valitydev/clickhouse-sink-connector
14
0
Fork 0
mirror of https://github.com/valitydev/clickhouse-sink-connector.git synced 2024-11-06 02:25:18 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Disabled configuration logging and changed clickhouse.server.pass to clickhouse.server.password so that it will be masked by debezium.

Browse Source
This commit is contained in:
kanthi Subramanian 2023-09-18 11:48:37 -04:00
parent aca39812a2
commit 63911fc92b
67 changed files with 111 additions and 107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@ -26,7 +26,7 @@ Single executable and lightweight.
Download the JAR file from the releases.
https://github.com/Altinity/clickhouse-sink-connector/releases
1. Update **MySQL information** in config.yaml: `database.hostname`, `database.port`, `database.user` and `database.password`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.pass`, `clickhouse.server.port`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.password`, `clickhouse.server.port`.
Also Update **ClickHouse information** for the following fields that are used to store the offset information- `offset.storage.jdbc.url`, `offset.storage.jdbc.user`, `offset.storage.jdbc.password`, `schema.history.internal.jdbc.url`, `schema.history.internal.jdbc.user`, and `schema.history.internal.jdbc.password`.
3. Update MySQL databases to be replicated: `database.include.list`.
4. Add table filters: `table.include.list`.
@ -47,7 +47,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -95,7 +95,7 @@ plugin.name: "pgoutput"
table.include.list: "public.tm"
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -215,7 +215,7 @@ GLOBAL OPTIONS:
| table.include.list | List of tables to be included in replication. |
| clickhouse.server.url | ClickHouse URL, For TLS(use `https` and set port to `8443`) |
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.pass | ClickHouse password |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port, For TLS(use the correct port `8443` or `443` |
| clickhouse.server.database | ClickHouse destination database |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.\<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ <br/>PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode <br/> MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
@ -365,7 +365,7 @@ mvn install -DskipTests=true
| topics | No | The list of topics. topics or topics.regex has to be provided. |
| clickhouse.server.url | | ClickHouse Server URL |
| clickhouse.server.user | | ClickHouse Server username |
| clickhouse.server.pass | | ClickHouse Server password |
| clickhouse.server.password | | ClickHouse Server password |
| clickhouse.server.database | | ClickHouse Database name |
| clickhouse.server.port | 8123 | ClickHouse Server port |
| clickhouse.topic2table.map | No | Map of Kafka topics to table names, <topic_name1>:<table_name1>,<topic_name2>:<table_name2> This variable will override the default mapping of topics to table names. |

2
doc/sink_configuration.md
View File

@ -25,7 +25,7 @@ This is a sample configuration that's used in creating the connector using the K
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
docs/assets/js/23374ca6.bac2875c.js
View File

File diff suppressed because one or more lines are too long

2
docs/assets/js/6ba0c277.5d828d25.js
View File

File diff suppressed because one or more lines are too long

2
docs/assets/js/b90b7e74.16f270e4.js
View File

File diff suppressed because one or more lines are too long

2
docs/docs/doc/sink_configuration/index.html
View File

File diff suppressed because one or more lines are too long

6
docs/docs/index.html
View File

File diff suppressed because one or more lines are too long

6
docs/docs/sink-connector-lightweight/index.html
View File

File diff suppressed because one or more lines are too long

6
docusaurus/clickhouse-sink-connector/build/Sink_Connector_Lightweight/index.html
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/0e384e19.cc4739c6.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/abe81e9a.41fe056f.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/b90b7e74.5dcda56d.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/d521f601.d7f18035.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/f46a48a4.6e828ba5.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/assets/js/fd91c785.edf96642.js
View File

File diff suppressed because one or more lines are too long

2
docusaurus/clickhouse-sink-connector/build/doc/sink_configuration/index.html
View File

File diff suppressed because one or more lines are too long

6
docusaurus/clickhouse-sink-connector/build/intro/index.html
View File

File diff suppressed because one or more lines are too long

6
docusaurus/clickhouse-sink-connector/build/sink-connector-lightweight/MySQL/index.html
View File

File diff suppressed because one or more lines are too long

6
docusaurus/clickhouse-sink-connector/build/sink-connector-lightweight/PostgreSQL/index.html
View File

File diff suppressed because one or more lines are too long

6
docusaurus/clickhouse-sink-connector/build/sink-connector-lightweight/index.html
View File

File diff suppressed because one or more lines are too long

10
docusaurus/clickhouse-sink-connector/docs/Sink_Connector_Lightweight.md
View File

@ -16,7 +16,7 @@ Single executable and lightweight.
Download the JAR file from the releases.
https://github.com/Altinity/clickhouse-sink-connector/releases
1. Update **MySQL information** in config.yaml: `database.hostname`, `database.port`, `database.user` and `database.password`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.pass`, `clickhouse.server.port`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.password`, `clickhouse.server.port`.
Also Update **ClickHouse information** for the following fields that are used to store the offset information- `offset.storage.jdbc.url`, `offset.storage.jdbc.user`, `offset.storage.jdbc.password`, `schema.history.internal.jdbc.url`, `schema.history.internal.jdbc.user`, and `schema.history.internal.jdbc.password`.
3. Update MySQL databases to be replicated: `database.include.list`.
4. Add table filters: `table.include.list`.
@ -34,7 +34,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -82,7 +82,7 @@ plugin.name: "pgoutput"
table.include.list: "public.tm"
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -196,7 +196,7 @@ GLOBAL OPTIONS:
| table.include.list | List of tables to be included in replication. |
| clickhouse.server.url | ClickHouse URL |
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.pass | ClickHouse password |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port |
| clickhouse.server.database | ClickHouse destination database |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.\<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ <br/>PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode <br/> MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
@ -345,7 +345,7 @@ mvn install -DskipTests=true
| topics | No | The list of topics. topics or topics.regex has to be provided. |
| clickhouse.server.url | | ClickHouse Server URL |
| clickhouse.server.user | | ClickHouse Server username |
| clickhouse.server.pass | | ClickHouse Server password |
| clickhouse.server.password | | ClickHouse Server password |
| clickhouse.server.database | | ClickHouse Database name |
| clickhouse.server.port | 8123 | ClickHouse Server port |
| clickhouse.topic2table.map | No | Map of Kafka topics to table names, <topic_name1>:<table_name1>,<topic_name2>:<table_name2> This variable will override the default mapping of topics to table names. |

2
docusaurus/clickhouse-sink-connector/docs/doc/sink_configuration.md
View File

@ -25,7 +25,7 @@ This is a sample configuration that's used in creating the connector using the K
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

10
docusaurus/clickhouse-sink-connector/docs/intro.md
View File

@ -16,7 +16,7 @@ Single executable and lightweight.
Download the JAR file from the releases.
https://github.com/Altinity/clickhouse-sink-connector/releases
1. Update **MySQL information** in config.yaml: `database.hostname`, `database.port`, `database.user` and `database.password`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.pass`, `clickhouse.server.port`.
2. Update **ClickHouse information** in config.yaml: `clickhouse.server.url`, `clickhouse.server.user`, `clickhouse.server.password`, `clickhouse.server.port`.
Also Update **ClickHouse information** for the following fields that are used to store the offset information- `offset.storage.jdbc.url`, `offset.storage.jdbc.user`, `offset.storage.jdbc.password`, `schema.history.internal.jdbc.url`, `schema.history.internal.jdbc.user`, and `schema.history.internal.jdbc.password`.
3. Update MySQL databases to be replicated: `database.include.list`.
4. Add table filters: `table.include.list`.
@ -34,7 +34,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -82,7 +82,7 @@ plugin.name: "pgoutput"
table.include.list: "public.tm"
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -196,7 +196,7 @@ GLOBAL OPTIONS:
| table.include.list | List of tables to be included in replication. |
| clickhouse.server.url | ClickHouse URL |
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.pass | ClickHouse password |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port |
| clickhouse.server.database | ClickHouse destination database |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.\<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ <br/>PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode <br/> MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
@ -345,7 +345,7 @@ mvn install -DskipTests=true
| topics | No | The list of topics. topics or topics.regex has to be provided. |
| clickhouse.server.url | | ClickHouse Server URL |
| clickhouse.server.user | | ClickHouse Server username |
| clickhouse.server.pass | | ClickHouse Server password |
| clickhouse.server.password | | ClickHouse Server password |
| clickhouse.server.database | | ClickHouse Database name |
| clickhouse.server.port | 8123 | ClickHouse Server port |
| clickhouse.topic2table.map | No | Map of Kafka topics to table names, <topic_name1>:<table_name1>,<topic_name2>:<table_name2> This variable will override the default mapping of topics to table names. |

8
docusaurus/clickhouse-sink-connector/docs/sink-connector-lightweight/MySQL/MySQL.md
View File

@ -17,7 +17,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -75,7 +75,7 @@ export database.password="root"
export snapshot.mode="schema_only"
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export connector.class="io.debezium.connector.postgresql.PostgresConnector"
export plugin.name="pgoutput"
@ -116,7 +116,7 @@ export auto.create.tables="true"
export clickhouse.server.url="clickhouse"
export clickhouse.server.port="8123"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.database="project"
export replacingmergetree.delete.column="_sign"
export metrics.port="8087"
@ -167,7 +167,7 @@ Update the MySQL/PostgreSQL and ClickHouse configuration values
value: "8123"
- name: clickhouse.server.user
value: "default"
- name: clickhouse.server.pass
- name: clickhouse.server.password
value: "2"
- name: clickhouse.server.database
value: "public"

8
docusaurus/clickhouse-sink-connector/docs/sink-connector-lightweight/PostgreSQL/PostgreSQL.md
View File

@ -17,7 +17,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -75,7 +75,7 @@ export database.password="root"
export snapshot.mode="schema_only"
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export connector.class="io.debezium.connector.postgresql.PostgresConnector"
export plugin.name="pgoutput"
@ -116,7 +116,7 @@ export auto.create.tables="true"
export clickhouse.server.url="clickhouse"
export clickhouse.server.port="8123"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.database="project"
export replacingmergetree.delete.column="_sign"
export metrics.port="8087"
@ -167,7 +167,7 @@ Update the MySQL/PostgreSQL and ClickHouse configuration values
value: "8123"
- name: clickhouse.server.user
value: "default"
- name: clickhouse.server.pass
- name: clickhouse.server.password
value: "2"
- name: clickhouse.server.database
value: "public"

8
docusaurus/clickhouse-sink-connector/docs/sink-connector-lightweight/sink-connector-lightweight.md
View File

@ -17,7 +17,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -75,7 +75,7 @@ export database.password="root"
export snapshot.mode="schema_only"
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export connector.class="io.debezium.connector.postgresql.PostgresConnector"
export plugin.name="pgoutput"
@ -116,7 +116,7 @@ export auto.create.tables="true"
export clickhouse.server.url="clickhouse"
export clickhouse.server.port="8123"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.database="project"
export replacingmergetree.delete.column="_sign"
export metrics.port="8087"
@ -167,7 +167,7 @@ Update the MySQL/PostgreSQL and ClickHouse configuration values
value: "8123"
- name: clickhouse.server.user
value: "default"
- name: clickhouse.server.pass
- name: clickhouse.server.password
value: "2"
- name: clickhouse.server.database
value: "public"

BIN
sink-connector-client/sink-connector-client
View File

Binary file not shown.

8
sink-connector-lightweight/README.md
View File

@ -17,7 +17,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
@ -75,7 +75,7 @@ export database.password="root"
export snapshot.mode="schema_only"
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export connector.class="io.debezium.connector.postgresql.PostgresConnector"
export plugin.name="pgoutput"
@ -116,7 +116,7 @@ export auto.create.tables="true"
export clickhouse.server.url="clickhouse"
export clickhouse.server.port="8123"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.database="project"
export replacingmergetree.delete.column="_sign"
export metrics.port="8087"
@ -167,7 +167,7 @@ Update the MySQL/PostgreSQL and ClickHouse configuration values
value: "8123"
- name: clickhouse.server.user
value: "default"
- name: clickhouse.server.pass
- name: clickhouse.server.password
value: "2"
- name: clickhouse.server.database
value: "public"

2
sink-connector-lightweight/config.examples/mysql_config.yaml
View File

@ -6,7 +6,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "localhost"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/docker/config.yml
View File

@ -8,7 +8,7 @@ database.include.list: test
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/docker/config_local.yml
View File

@ -6,7 +6,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "localhost"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/docker/config_postgres.yml
View File

@ -9,7 +9,7 @@ plugin.name: "pgoutput"
table.include.list: "public.tm,public.tm2"
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

4
sink-connector-lightweight/docker/docker-compose-postgres.yml
View File

@ -60,6 +60,10 @@ services:
- docker_postgres.env
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
JAVA_OPTS: >
-Xmx5G
-Xms128m
volumes:
#- ./data:/data
- ./config_postgres.yml:/config.yml

2
sink-connector-lightweight/docker/docker.env
View File

@ -13,7 +13,7 @@ export replacingmergetree.delete.column="_sign"
export metrics.port=8083
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export thread.pool.size="10"
export provide.transaction.metadata="false"

2
sink-connector-lightweight/docker/docker_mongo.env
View File

@ -15,7 +15,7 @@ export auto.create.tables="true"
export clickhouse.server.url="clickhouse"
export clickhouse.server.port="8123"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.database="project"
export replacingmergetree.delete.column="_sign"
export metrics.port="8087"

2
sink-connector-lightweight/docker/docker_postgres.env
View File

@ -12,7 +12,7 @@ export replacingmergetree.delete.column="_sign"
export metrics.port=8087
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export clickhouse.server.database="test"
export offset.storage="org.apache.kafka.connect.storage.FileOffsetBackingStore"

2
sink-connector-lightweight/docker/local.env
View File

@ -11,5 +11,5 @@ export replacingmergetree.delete.column="_sign"
export metrics.port=8083
export clickhouse.server.url="localhost"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"

2
sink-connector-lightweight/helm/clickhouse-debezium-embedded/values.yaml
View File

@ -124,7 +124,7 @@ env:
value: "8123"
- name: clickhouse.server.user
value: "default"
- name: clickhouse.server.pass
- name: clickhouse.server.password
value: "2"
- name: clickhouse.server.database
value: "public"

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ClickHouseDebeziumEmbeddedMongoIT.java
View File

@ -106,7 +106,7 @@
// defaultProps.setProperty("clickhouse.server.url", clickHouseContainer.getHost());
// defaultProps.setProperty("clickhouse.server.port", String.valueOf(clickHouseContainer.getFirstMappedPort()));
// defaultProps.setProperty("clickhouse.server.user", "default");
// defaultProps.setProperty("clickhouse.server.pass", "");
// defaultProps.setProperty("clickhouse.server.password", "");
// defaultProps.setProperty("clickhouse.server.database", "project");
// defaultProps.setProperty("replacingmergetree.delete.column", "_sign");
// defaultProps.setProperty("metrics.port", "8087");

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ClickHouseDebeziumEmbeddedPostgreSQLIT.java
View File

@ -77,7 +77,7 @@ public class ClickHouseDebeziumEmbeddedPostgreSQLIT {
defaultProps.setProperty("clickhouse.server.url", clickHouseContainer.getHost());
defaultProps.setProperty("clickhouse.server.port", String.valueOf(clickHouseContainer.getFirstMappedPort()));
defaultProps.setProperty("clickhouse.server.user", "default");
defaultProps.setProperty("clickhouse.server.pass", "");
defaultProps.setProperty("clickhouse.server.password", "");
defaultProps.setProperty("clickhouse.server.database", "public");
defaultProps.setProperty("replacingmergetree.delete.column", "_sign");
defaultProps.setProperty("metrics.port", "8087");

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ClickHouseDebeziumEmbeddedPostgresDecoderBufsDockerIT.java
View File

@ -73,7 +73,7 @@ public class ClickHouseDebeziumEmbeddedPostgresDecoderBufsDockerIT {
properties.put("clickhouse.server.url", "clickhouse");
properties.put("clickhouse.server.port", "8123");
properties.put("clickhouse.server.user", "default");
properties.put("clickhouse.server.pass", "");
properties.put("clickhouse.server.password", "");
properties.put("clickhouse.server.database", "public");
properties.put("replacingmergetree.delete.column", "_sign");
properties.put("metrics.port", "8087");

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ClickHouseDebeziumEmbeddedPostgresDockerIT.java
View File

@ -70,7 +70,7 @@ public class ClickHouseDebeziumEmbeddedPostgresDockerIT {
properties.put("clickhouse.server.url", "clickhouse");
properties.put("clickhouse.server.port", "8123");
properties.put("clickhouse.server.user", "default");
properties.put("clickhouse.server.pass", "");
properties.put("clickhouse.server.password", "");
properties.put("clickhouse.server.database", "public");
properties.put("replacingmergetree.delete.column", "_sign");
properties.put("metrics.port", "8087");

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/config/AppConfigurationTest.java
View File

@ -18,7 +18,7 @@ public class AppConfigurationTest {
@SetEnvironmentVariable(key = "database.pass", value = "password")
@SetEnvironmentVariable(key = "clickhouse.server.url", value = "localhost")
@SetEnvironmentVariable(key = "clickhouse.server.user", value = "root")
@SetEnvironmentVariable(key = "clickhouse.server.pass", value = "root")
@SetEnvironmentVariable(key = "clickhouse.server.password", value = "root")
@SetEnvironmentVariable(key = "clickhouse.server.port", value = "8123")
@SetEnvironmentVariable(key = "snapshot.mode", value = "schema_only")
public void testParseConfiguration() throws Exception {

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/config/mysql_config.yaml
View File

@ -6,7 +6,7 @@ database.include.list: sbtest
#table.include.list=sbtest1
clickhouse.server.url: "localhost"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ddl/parser/ClickHouseDebeziumEmbeddedDDLBaseIT.java
View File

@ -101,7 +101,7 @@ public class ClickHouseDebeziumEmbeddedDDLBaseIT {
defaultProps.setProperty("clickhouse.server.url", clickHouseContainer.getHost());
defaultProps.setProperty("clickhouse.server.port", String.valueOf(clickHouseContainer.getFirstMappedPort()));
defaultProps.setProperty("clickhouse.server.user", "default");
defaultProps.setProperty("clickhouse.server.pass", "");
defaultProps.setProperty("clickhouse.server.password", "");
defaultProps.setProperty("clickhouse.server.database", "employees");
defaultProps.setProperty("offset.storage.jdbc.url", String.format("jdbc:clickhouse://%s:%s",

2
sink-connector-lightweight/src/test/java/com/altinity/clickhouse/debezium/embedded/ddl/parser/ClickHouseDebeziumEmbeddedDDLTableOperationsIT.java
View File

@ -265,7 +265,7 @@ public class ClickHouseDebeziumEmbeddedDDLTableOperationsIT {
defaultProps.setProperty("clickhouse.server.url", clickHouseContainer.getHost());
defaultProps.setProperty("clickhouse.server.port", String.valueOf(clickHouseContainer.getFirstMappedPort()));
defaultProps.setProperty("clickhouse.server.user", "default");
defaultProps.setProperty("clickhouse.server.pass", "");
defaultProps.setProperty("clickhouse.server.password", "");
defaultProps.setProperty("clickhouse.server.database", "employees");
defaultProps.setProperty("replacingmergetree.delete.column", "_sign");
defaultProps.setProperty("metrics.port", "8088");

2
sink-connector-lightweight/src/test/resources/config.yml
View File

@ -8,7 +8,7 @@ database.include.list: employees
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "default"
clickhouse.server.pass: ""
clickhouse.server.password: ""
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/tests/integration/env/auto/config.yml vendored
View File

@ -8,7 +8,7 @@ database.include.list: test
#table.include.list=sbtest1
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.password: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"

2
sink-connector-lightweight/tests/integration/env/auto/docker.env vendored
View File

@ -12,7 +12,7 @@ export replacingmergetree.delete.column="_sign"
export metrics.port=8083
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export thread.pool.size="10"
export provide.transaction.metadata="false"

2
sink-connector-lightweight/tests/integration/env/manual/docker.env vendored
View File

@ -12,7 +12,7 @@ export replacingmergetree.delete.column="_sign"
export metrics.port=8083
export clickhouse.server.url="clickhouse"
export clickhouse.server.user="root"
export clickhouse.server.pass="root"
export clickhouse.server.password="root"
export clickhouse.server.port="8123"
export thread.pool.size="10"
export provide.transaction.metadata="false"

2
sink-connector-lightweight/tests/integration/tests/manual_scripts/sink-connector-setup-schema-registry-local.sh
View File

@ -35,7 +35,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector-lightweight/tests/integration/tests/manual_scripts/sink-connector-setup-schema-registry.sh
View File

@ -34,7 +34,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector-lightweight/tests/integration/tests/manual_scripts/sink-connector-setup.sh
View File

@ -27,7 +27,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"buffer.count.records": "${BUFFER_COUNT}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

4
sink-connector-lightweight/tests/integration/tests/steps/service_settings_steps.py
View File

@ -32,7 +32,7 @@ def init_sink_connector(
"clickhouse.topic2table.map": "SERVER5432.test.users:users",
"clickhouse.server.url": "clickhouse",
"clickhouse.server.user": "root",
"clickhouse.server.pass": "root",
"clickhouse.server.password": "root",
"clickhouse.server.database": "test",
"clickhouse.server.port": 8123,
"clickhouse.table.name": "users",
@ -83,7 +83,7 @@ EOF"""
"clickhouse.topic2table.map": "",
"clickhouse.server.url": "clickhouse",
"clickhouse.server.user": "root",
"clickhouse.server.pass": "root",
"clickhouse.server.password": "root",
"clickhouse.server.database": "test",
"clickhouse.server.port": 8123,
"clickhouse.table.name": "users",

2
sink-connector/deploy/helm/altinity-sink-connector/templates/sink-kafkaconnector.yaml
View File

@ -14,7 +14,7 @@ spec:
topics: {{ $sinkConnector.topics }}
clickhouse.server.url: {{ $.Values.clickhouseSinkConnector.clickhouse.server.url }}
clickhouse.server.user: {{ $.Values.clickhouseSinkConnector.clickhouse.server.user }}
clickhouse.server.pass: {{ $.Values.clickhouseSinkConnector.clickhouse.server.pass }}
clickhouse.server.password: {{ $.Values.clickhouseSinkConnector.clickhouse.server.password }}
clickhouse.server.database: {{ $.Values.clickhouseSinkConnector.clickhouse.server.url }}
clickhouse.server.port: {{ $.Values.clickhouseSinkConnector.clickhouse.server.port }}
clickhouse.topic2table.map: "employees:employee"

2
sink-connector/deploy/k8s/sink-connector-apicurio-avro.yaml
View File

@ -17,7 +17,7 @@ spec:
clickhouse.topic2table.map: "${TOPICS_TABLE_MAP}"
clickhouse.server.url: "${CLICKHOUSE_HOST}"
clickhouse.server.user: "${CLICKHOUSE_USER}"
clickhouse.server.pass: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.password: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.database: "${CLICKHOUSE_DATABASE}"
clickhouse.server.port: "${CLICKHOUSE_PORT}"
clickhouse.table.name: "${CLICKHOUSE_TABLE}"

2
sink-connector/deploy/k8s/sink-connector-avro.yaml
View File

@ -17,7 +17,7 @@ spec:
clickhouse.topic2table.map: "${TOPICS_TABLE_MAP}"
clickhouse.server.url: "${CLICKHOUSE_HOST}"
clickhouse.server.user: "${CLICKHOUSE_USER}"
clickhouse.server.pass: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.password: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.database: "${CLICKHOUSE_DATABASE}"
clickhouse.server.port: "${CLICKHOUSE_PORT}"
clickhouse.table.name: "${CLICKHOUSE_TABLE}"

2
sink-connector/deploy/k8s/sink-connector-json.yaml
View File

@ -18,7 +18,7 @@ spec:
clickhouse.topic2table.map: "${TOPICS_TABLE_MAP}"
clickhouse.server.url: "${CLICKHOUSE_HOST}"
clickhouse.server.user: "${CLICKHOUSE_USER}"
clickhouse.server.pass: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.password: "${CLICKHOUSE_PASSWORD}"
clickhouse.server.database: "${CLICKHOUSE_DATABASE}"
clickhouse.server.port: "${CLICKHOUSE_PORT}"
clickhouse.table.name: "${CLICKHOUSE_TABLE}"

4
sink-connector/deploy/sink-connector-setup-database.sh
View File

@ -31,7 +31,7 @@ if [[ $1 == "apicurio" ]]; then
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
# "clickhouse.table.name": "${CLICKHOUSE_TABLE}",
@ -80,7 +80,7 @@ else
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

4
sink-connector/deploy/sink-connector-setup-schema-registry.sh
View File

@ -42,7 +42,7 @@ if [[ $2 == "apicurio" ]]; then
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",
@ -91,7 +91,7 @@ else
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector/deploy/sink-connector-setup.sh
View File

@ -27,7 +27,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"buffer.count.records": "${BUFFER_COUNT}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector/src/main/java/com/altinity/clickhouse/sink/connector/ClickHouseSinkConnectorConfig.java
View File

@ -41,7 +41,7 @@ public class ClickHouseSinkConnectorConfig extends AbstractConfig {
}
protected ClickHouseSinkConnectorConfig(ConfigDef config, Map<String, String> properties) {
super(config, properties);
super(config, properties, false);
}
/**

2
sink-connector/src/main/java/com/altinity/clickhouse/sink/connector/ClickHouseSinkConnectorConfigVariables.java
View File

@ -12,7 +12,7 @@ public enum ClickHouseSinkConnectorConfigVariables {
CLICKHOUSE_URL("clickhouse.server.url"),
CLICKHOUSE_USER("clickhouse.server.user"),
CLICKHOUSE_PASS("clickhouse.server.pass"),
CLICKHOUSE_PASS("clickhouse.server.password"),
CLICKHOUSE_DATABASE("clickhouse.server.database"),
CLICKHOUSE_PORT("clickhouse.server.port"),

2
sink-connector/tests/integration/tests/manual_scripts/sink-connector-setup-schema-registry-local.sh
View File

@ -35,7 +35,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector/tests/integration/tests/manual_scripts/sink-connector-setup-schema-registry.sh
View File

@ -34,7 +34,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

2
sink-connector/tests/integration/tests/manual_scripts/sink-connector-setup.sh
View File

@ -27,7 +27,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
"buffer.count.records": "${BUFFER_COUNT}",
"clickhouse.server.url": "${CLICKHOUSE_HOST}",
"clickhouse.server.user": "${CLICKHOUSE_USER}",
"clickhouse.server.pass": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.password": "${CLICKHOUSE_PASSWORD}",
"clickhouse.server.database": "${CLICKHOUSE_DATABASE}",
"clickhouse.server.port": ${CLICKHOUSE_PORT},
"clickhouse.table.name": "${CLICKHOUSE_TABLE}",

4
sink-connector/tests/integration/tests/steps/service_settings_steps.py
View File

@ -32,7 +32,7 @@ def init_sink_connector(
"clickhouse.topic2table.map": "SERVER5432.test.users:users",
"clickhouse.server.url": "clickhouse",
"clickhouse.server.user": "root",
"clickhouse.server.pass": "root",
"clickhouse.server.password": "root",
"clickhouse.server.database": "test",
"clickhouse.server.port": 8123,
"clickhouse.table.name": "users",
@ -83,7 +83,7 @@ EOF"""
"clickhouse.topic2table.map": "",
"clickhouse.server.url": "clickhouse",
"clickhouse.server.user": "root",
"clickhouse.server.pass": "root",
"clickhouse.server.password": "root",
"clickhouse.server.database": "test",
"clickhouse.server.port": 8123,
"clickhouse.table.name": "users",