Commit Graph

11 Commits

Author SHA1 Message Date
Kanthi Subramanian
63252d82d4 Enable transactions metadata support in config. 2022-05-10 17:54:28 -04:00
Kanthi Subramanian
ff71203377 Fixed bug with kafka metadata columns not added when there is a source, CH columns mismatch. Added unittest for QueryFormatter class. 2022-05-10 15:59:53 -04:00
Kanthi Subramanian
b0f440ab2d Added more source kafka metadata columns(binlog pos, server id, thread, source ts_ms). 2022-05-10 12:58:54 -04:00
Kanthi Subramanian
d84198609b Added logic to use micrometer library and expose prometheus port for writing metrics. Added test case for deduplication. 2022-05-03 22:26:57 -04:00
Kanthi Subramanian
315d147296 Sink connector configuration documentation. 2022-04-29 21:00:54 -04:00
Kanthi Subramanian
b3c967ae91 Added logic to automatically map table name to topic name if not provided in the configuration. Added python test to query clickhouse using clickhouse python library. 2022-04-27 17:16:37 -04:00
Kanthi Subramanian
671a0f6f84 Added test case in python to test out of range date ranges. 2022-04-27 13:58:27 -04:00
Kanthi Subramanian
57ec2e84d8 Changed batch insert input function query syntax. Added logic to insert fake data for Products table to test multiple topics ingestion. 2022-04-26 22:17:48 -04:00
Kanthi Subramanian
96a81778cd Added support for multiple tasks in Sink. 2022-04-24 20:04:15 -04:00
Kanthi Subramanian
5c9c1cd55c Added logic to insert fake data to mysql using Faker library. 2022-04-22 16:55:02 -04:00
Kanthi Subramanian
5beef400ef Added python test framework to setup mysql connection. 2022-04-21 16:50:17 -04:00