Kanthi Subramanian
|
78634b19fa
|
Added functions to extract version and sign columns from engine column in system tables.
|
2022-06-03 13:15:09 -04:00 |
|
Kanthi Subramanian
|
2a50183b28
|
Updated docker image to dockerhub for sink connector
|
2022-06-03 10:27:43 -04:00 |
|
Kanthi Subramanian
|
900c2e72f3
|
Added logic to read version column and to set it to timestamp(epoch in ms) for updates.
|
2022-06-01 17:18:44 -04:00 |
|
Kanthi Subramanian
|
d174c68f3e
|
Set flag in debezium connector to fix public key retrieval error.
|
2022-06-01 12:52:52 -04:00 |
|
Kanthi Subramanian
|
98fad54edc
|
Fixed bug with adding batch for insert statements.
|
2022-06-01 10:17:55 -04:00 |
|
Kanthi Subramanian
|
eebddf9b5f
|
Fixed quotes in get table engine from system table.
|
2022-05-31 18:11:11 -04:00 |
|
Kanthi Subramanian
|
a8cc587553
|
Fixed logic of setting sign column for updates.
|
2022-05-31 16:00:49 -04:00 |
|
Kanthi Subramanian
|
b36dbac88d
|
Updated pre-requisites for local setup
|
2022-05-31 13:29:39 -04:00 |
|
Kanthi Subramanian
|
c39844cd65
|
Refactoring to support retrieving before/after columns and logic to insert records(before and after separately) when its an update.
|
2022-05-31 11:18:14 -04:00 |
|
Kanthi Subramanian
|
456fbb0c8a
|
Added function to identify Clickhouse table engine using system tables.
|
2022-05-28 10:37:53 -04:00 |
|
Kanthi Subramanian
|
1122af5bbb
|
Added function to identify Clickhouse table engine(CollapsingMergeTree or ReplacingMergeTree)
|
2022-05-27 13:51:04 -04:00 |
|
Kanthi Subramanian
|
690334f3a7
|
Changed the sign column to a Sink configuration variable.
|
2022-05-26 16:29:26 -04:00 |
|
Kanthi Subramanian
|
5d38acd39c
|
Refactoring, Added class for storing kafka offsets.
|
2022-05-24 20:45:32 -04:00 |
|
Kanthi Subramanian
|
9c10224e25
|
Refactoring, moving code to packages.
|
2022-05-24 15:55:25 -04:00 |
|
Kanthi Subramanian
|
0d3a29dbc3
|
Updated Local setup instructions.
|
2022-05-24 09:50:03 -04:00 |
|
Kanthi Subramanian
|
e8ed301958
|
Added logic to parse delete CDC record and set sign flag to -1.
|
2022-05-23 13:35:33 -04:00 |
|
Kanthi Subramanian
|
c5c5ed558d
|
Added grafana datasource yaml file.
|
2022-05-22 20:26:59 -04:00 |
|
Kanthi Subramanian
|
22d4923d60
|
Fixed bug with storing raw data.
|
2022-05-20 10:26:43 -04:00 |
|
Kanthi Subramanian
|
e45b0f27a9
|
Added grafana dashboard, Add mysql snap tests to run load tests, Added logic to register source and consumer lag metrics to prometheus.
|
2022-05-17 10:23:14 -04:00 |
|
Kanthi Subramanian
|
eace9edb1d
|
Added test case for updating records.
|
2022-05-16 10:53:06 -04:00 |
|
Kanthi Subramanian
|
cba499d267
|
Refactor Deduplicator to accommodate multiple topics.
|
2022-05-14 22:58:47 -04:00 |
|
Kanthi Subramanian
|
8dacfea78f
|
Merge remote-tracking branch 'origin/main'
|
2022-05-13 13:34:03 -04:00 |
|
Kanthi Subramanian
|
d9a7ee532e
|
Fixed bug with adding kafka metadata columns.
|
2022-05-13 13:33:56 -04:00 |
|
Vladislav Klimenko
|
dd01674b14
|
extract kubectl finalization
|
2022-05-12 17:20:16 +03:00 |
|
Vladislav Klimenko
|
6e76e34172
|
make port forward as a background process
|
2022-05-12 17:18:35 +03:00 |
|
Vladislav Klimenko
|
29f2454106
|
polish pipeline
|
2022-05-12 17:09:38 +03:00 |
|
Vladislav Klimenko
|
ebb0af59b9
|
adjust port forwarder
|
2022-05-12 17:06:20 +03:00 |
|
Vladislav Klimenko
|
7e2c18f134
|
extract port forward
|
2022-05-12 16:57:17 +03:00 |
|
Vladislav Klimenko
|
d2c8f37270
|
load to local registry
|
2022-05-12 16:57:05 +03:00 |
|
Vladislav Klimenko
|
ad29db6c2b
|
use localhost for altinity connectora
|
2022-05-12 16:56:49 +03:00 |
|
Vladislav Klimenko
|
b539882ad1
|
move unused files
|
2022-05-12 16:56:23 +03:00 |
|
Vladislav Klimenko
|
d059502963
|
docs: do not pull altinity images yet
|
2022-05-12 11:17:10 +03:00 |
|
Vladislav Klimenko
|
7510697231
|
use altinity image directly
|
2022-05-12 11:16:44 +03:00 |
|
Kanthi Subramanian
|
ed53f84626
|
Fixed image name in github action for building docker images.
|
2022-05-11 21:09:38 -04:00 |
|
Kanthi Subramanian
|
b2989fee91
|
Add mysql master/slave replication setup for docker-compose. Updated debezium source connector config to connect to mysql slave.
|
2022-05-11 21:01:12 -04:00 |
|
Vladislav Klimenko
|
6c39b8e78c
|
use graalvm image
|
2022-05-11 15:05:55 +03:00 |
|
Vladislav Klimenko
|
3a4bb90362
|
image name
|
2022-05-11 15:05:43 +03:00 |
|
Vladislav Klimenko
|
df1835c6f3
|
make image names for readbale
|
2022-05-11 15:03:51 +03:00 |
|
Vladislav Klimenko
|
4061fc7607
|
add graalvm builder
|
2022-05-11 14:56:04 +03:00 |
|
Vladislav Klimenko
|
d5036b4d26
|
use proper files
|
2022-05-11 14:55:52 +03:00 |
|
Vladislav Klimenko
|
538b407322
|
unify naming and base image
|
2022-05-11 14:55:16 +03:00 |
|
Kanthi Subramanian
|
63252d82d4
|
Enable transactions metadata support in config.
|
2022-05-10 17:54:28 -04:00 |
|
Kanthi Subramanian
|
ff71203377
|
Fixed bug with kafka metadata columns not added when there is a source, CH columns mismatch. Added unittest for QueryFormatter class.
|
2022-05-10 15:59:53 -04:00 |
|
Kanthi Subramanian
|
b0f440ab2d
|
Added more source kafka metadata columns(binlog pos, server id, thread, source ts_ms).
|
2022-05-10 12:58:54 -04:00 |
|
Kanthi Subramanian
|
82b23f2e5d
|
Added test data for all blob types.
|
2022-05-09 13:58:02 -04:00 |
|
Kanthi Subramanian
|
7648df357d
|
Added test data for all blob types. Changed input function query to support cases when there are more MySQL columns compared to Clickhouse columns.
|
2022-05-09 13:56:11 -04:00 |
|
Kanthi Subramanian
|
96dc94367a
|
Added GraalVM kafka connect image.
|
2022-05-08 09:28:05 -04:00 |
|
Kanthi Subramanian
|
bf8604d04b
|
Added task id to prometheus metrics.
|
2022-05-07 17:20:15 -04:00 |
|
Kanthi Subramanian
|
995f1a14d0
|
Merge remote-tracking branch 'origin/main'
|
2022-05-06 14:21:23 -04:00 |
|
Kanthi Subramanian
|
a7a97df488
|
Fixed logic with storing min offsets.
|
2022-05-06 14:21:16 -04:00 |
|