valitydev/clickhouse-sink-connector
14
0
Fork 0
mirror of https://github.com/valitydev/clickhouse-sink-connector.git synced 2024-11-06 18:45:27 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
315 Commits 8 Branches 0 Tags 202 MiB
6ce8c4349d
Commit Graph

315 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kanthi Subramanian
63252d82d4 Enable transactions metadata support in config. 2022-05-10 17:54:28 -04:00
Kanthi Subramanian
ff71203377 Fixed bug with kafka metadata columns not added when there is a source, CH columns mismatch. Added unittest for QueryFormatter class. 2022-05-10 15:59:53 -04:00
Kanthi Subramanian
b0f440ab2d Added more source kafka metadata columns(binlog pos, server id, thread, source ts_ms). 2022-05-10 12:58:54 -04:00
Kanthi Subramanian
82b23f2e5d Added test data for all blob types. 2022-05-09 13:58:02 -04:00
Kanthi Subramanian
7648df357d Added test data for all blob types. Changed input function query to support cases when there are more MySQL columns compared to Clickhouse columns. 2022-05-09 13:56:11 -04:00
Kanthi Subramanian
96dc94367a Added GraalVM kafka connect image. 2022-05-08 09:28:05 -04:00
Kanthi Subramanian
bf8604d04b Added task id to prometheus metrics. 2022-05-07 17:20:15 -04:00
Kanthi Subramanian
995f1a14d0 Merge remote-tracking branch 'origin/main' 2022-05-06 14:21:23 -04:00
Kanthi Subramanian
a7a97df488 Fixed logic with storing min offsets. 2022-05-06 14:21:16 -04:00
Vladislav Klimenko
2b01571692 split commands for copy+paste 2022-05-06 20:39:59 +03:00
Vladislav Klimenko
0a57412780 delete duplicates 2022-05-06 20:35:43 +03:00
Vladislav Klimenko
a38506654b Merge remote-tracking branch 'altinity/main' into main 2022-05-06 20:34:53 +03:00
Vladislav Klimenko
2641af59af no need to keep artefacts any more - will be built from sources 2022-05-06 20:34:28 +03:00
Vladislav Klimenko
a9d8a18b33 build images from sources 2022-05-06 20:33:43 +03:00
Vladislav Klimenko
cd9a12cfd2 do not keep libs, use them during docker build process 2022-05-06 20:33:09 +03:00
Vladislav Klimenko
f5c1bf066a use debezium-base suffix 2022-05-06 20:32:22 +03:00
Kanthi Subramanian
ae08242ec8 Added logic to also include partition in bulk metrics as one task could handle multiple partitions. 2022-05-06 12:25:33 -04:00
Kanthi Subramanian
f51591c63b Added config variables to enable metrics and to pass the metrics endpoint port. Refactor metrics class. 2022-05-06 11:41:31 -04:00
Kanthi Subramanian
b2eb67d2c0 Fix github runner for building docker images. 2022-05-06 10:01:35 -04:00
Kanthi Subramanian
06ed8b180f Added logic to record min and max offset of the bulk message in prometheus. 2022-05-06 09:57:30 -04:00
Kanthi Subramanian
3c8394ff07 Added test for spoofing sink records(From Bigquery), Fixed build script for docker-compose. 2022-05-06 09:17:52 -04:00
Vladislav Klimenko
ff4d1043d1 redpanda condition 2022-05-06 15:38:01 +03:00
Vladislav Klimenko
5d59cdb4db doc: move k8s docs to main folder 2022-05-06 15:14:20 +03:00
Vladislav Klimenko
fd67311834 doc: pipeline 2022-05-06 15:13:16 +03:00
Vladislav Klimenko
fb6a8e7db9 doc: add pipeline tools 2022-05-06 15:13:00 +03:00
Vladislav Klimenko
c05f512057 doc: extract build conneecto images 2022-05-06 15:12:42 +03:00
Vladislav Klimenko
94cca466a5 doc files 2022-05-06 15:12:16 +03:00
Vladislav Klimenko
e15cbcd559 add topic checker and registry scheker 2022-05-06 14:27:13 +03:00
Vladislav Klimenko
e63a800633 add rollouts for mysql + CH clusters 2022-05-06 14:21:16 +03:00
Vladislav Klimenko
28871da86a manual 2022-05-06 14:08:41 +03:00
Vladislav Klimenko
57520b3852 adapt connectors 2022-05-06 14:08:33 +03:00
Vladislav Klimenko
5b30446d61 extract minikube pre-load 2022-05-06 14:08:06 +03:00
Vladislav Klimenko
d87f3bee4a use strimzi-recommended build processes 2022-05-06 14:07:41 +03:00
Vladislav Klimenko
97495268ab adapt builder scripts 2022-05-06 14:07:20 +03:00
Kanthi Subramanian
4451438f71 Added logic to check if the column data type is DateTime64 for limiting the range. 2022-05-05 18:01:20 -04:00
Kanthi Subramanian
3d7fb07072 Added unit test case and logic to limit the DateTime values to clickhouse supported datetime ranges. 2022-05-05 12:13:42 -04:00
Kanthi Subramanian
e1cf390123 Added unit test case and logic to limit the Date values to clickhouse supported date ranges. 2022-05-05 11:09:05 -04:00
Vladislav Klimenko
0e5e1434dd docs k8s 2022-05-05 14:48:53 +03:00
Vladislav Klimenko
fcf69bf15b doc: extact minikube load images cache 2022-05-05 14:48:27 +03:00
Vladislav Klimenko
045cf1ac3f make delete executable 2022-05-05 14:48:00 +03:00
Vladislav Klimenko
667a71b939 extract avro from libs 2022-05-05 12:57:10 +03:00
Vladislav Klimenko
fe0cdf1a8b unify connecto manifests and add some logging 2022-05-05 12:10:03 +03:00
Vladislav Klimenko
c5365921ed jq tool 2022-05-05 02:09:27 +03:00
Vladislav Klimenko
9f3d164062 add sink rollout 2022-05-05 02:03:10 +03:00
Vladislav Klimenko
2995acaaa1 add autopacker 2022-05-05 00:42:20 +03:00
Kanthi Subramanian
cd23c15922 Merge remote-tracking branch 'origin/main' 2022-05-04 14:53:10 -04:00
Kanthi Subramanian
2e5627611a Updated Sink configuration documentation. 2022-05-04 14:52:27 -04:00
Vladislav Klimenko
474bc546d9 Merge remote-tracking branch 'altinity/main' into main 2022-05-04 19:10:29 +03:00
Vladislav Klimenko
aac6342af1 add k8s pipeline 2022-05-04 19:10:14 +03:00
Kanthi Subramanian
e67c9cd8e6 Fixed transformation of blob to string(hex). Changed prometheus reporter to use Counter instead of Gauge. 2022-05-04 11:21:11 -04:00