mirror of https://github.com/valitydev/clickhouse-sink-connector.git synced 2024-11-06 18:45:27 +00:00

Replicate data from MySQL, Postgres and MongoDB to ClickHouse

Go to file

Kanthi Subramanian df67e52d42 Refactoring KafkaOffsetWriter and added test case for MySQL JSON Data type.		2022-06-17 13:58:52 -04:00
.github/workflows	Fixed image name in github action for building docker images.	2022-05-11 21:09:38 -04:00
deploy	Added postgres docker compose setup, seed data for postgres and debezium connector configuration for PostgreSQL	2022-06-16 17:28:18 -04:00
doc	Added kafka lag dashboard	2022-06-10 21:41:09 -04:00
docker	Refactoring KafkaOffsetWriter and added test case for MySQL JSON Data type.	2022-06-17 13:58:52 -04:00
src	Refactoring KafkaOffsetWriter and added test case for MySQL JSON Data type.	2022-06-17 13:58:52 -04:00
tests	Refactoring KafkaOffsetWriter and added test case for MySQL JSON Data type.	2022-06-17 13:58:52 -04:00
.gitignore	Added initial commit of files from the original repo	2022-03-28 09:16:31 -04:00
LICENSE	Initial commit	2022-03-21 11:32:45 +03:00
pom.xml	Added logic to record min and max offset of the bulk message in prometheus.	2022-05-06 09:57:30 -04:00
README.md	Updated README to include postgres as source database.	2022-06-16 17:29:04 -04:00
strimzi.yml	Updated documentation with Data types mapping.	2022-04-29 13:32:53 -04:00

Altinity Sink Connector for ClickHouse

Sink connector sinks data from Kafka into Clickhouse. The connector is tested with the following converters

Features

Inserts, Updates and Deletes using ReplacingMergeTree/CollapsingMergeTreeUpdates/Deletes
Deduplication logic to dedupe records from Kafka topic.
Exactly once semantics
Bulk insert to Clickhouse.
Store Kafka metadata Kafka Metadata