2022-04-01 12:00:03 +00:00
|
|
|
# ClickHouse Sink Connector
|
|
|
|
|
|
2022-04-29 17:32:53 +00:00
|
|
|
Sink connector sinks data from Kafka into Clickhouse.
|
|
|
|
|
The connector is tested with the following converters
|
|
|
|
|
- JsonConverter
|
|
|
|
|
- AvroConverter (Using [Apicurio Schema Registry](https://www.apicur.io/registry/))
|
|
|
|
|
|
2022-05-06 16:25:33 +00:00
|
|
|
|
2022-06-09 13:53:48 +00:00
|
|
|
- Inserts, Updates and Deletes using ReplacingMergeTree/CollapsingMergeTree[Updates/Deletes](doc/mutable_data.md)
|
2022-05-06 16:25:33 +00:00
|
|
|
- Deduplication logic to dedupe records from Kafka topic.
|
|
|
|
|
- Exactly once semantics
|
|
|
|
|
- Bulk insert to Clickhouse.
|
2022-05-10 16:58:54 +00:00
|
|
|
- Store Kafka metadata [Kafka Metadata](doc/Kafka_metadata.md)
|
2022-04-01 12:00:03 +00:00
|
|
|
|
|
|
|
|
|
2022-04-29 17:32:53 +00:00
|
|
|
|
|
|
|
|
## Documentation
|
|
|
|
|
- [Data Types](doc/DataTypes.md)
|
|
|
|
|
- [Architecture](doc/architecture.md)
|
|
|
|
|
- [Local Setup](doc/setup.md)
|
|
|
|
|
- [Sink Configuration](doc/sink_configuration.md)
|
|
|
|
|
- [Testing](doc/TESTING.md)
|
|
|
|
|
- [Glossary](doc/glossary.md)
|
