Added document for SysBench setup.

deploy/sink-connector-setup-schema-registry.sh

@@ -24,7 +24,7 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
   "name": "${CONNECTOR_NAME}",
   "config": {
     "connector.class": "com.altinity.clickhouse.sink.connector.ClickHouseSinkConnector",
-    "tasks.max": "3",
+    "tasks.max": "6",
     "topics": "${TOPICS}",
     "clickhouse.topic2table.map": "${TOPICS_TABLE_MAP}",
     "clickhouse.server.url": "${CLICKHOUSE_HOST}",
@@ -44,14 +44,14 @@ cat <<EOF | curl --request POST --url "${CONNECTORS_MANAGEMENT_URL}" --header 'C
     "value.converter.apicurio.registry.auto-register": "true",
     "value.converter.apicurio.registry.find-latest": "true",
     "store.kafka.metadata": true,
-    "topic.creation.default.partitions": 1,
+    "topic.creation.default.partitions": 6,
 
     "store.raw.data": false,
     "store.raw.data.column": "raw_data",
 
     "metrics.enable": true,
     "metrics.port": 8084,
-    "buffer.flush.time": 10,
+    "buffer.flush.time": 1,
     "fetch.min.bytes": 52428800,
 
     "enable.kafka.offset": false

doc/Performance.md

@@ -0,0 +1,32 @@
+# Setup
+We use `SysBench` to perform load testing.
+https://github.com/akopytov/sysbench
+
+To Run SysBench tests, run the `debezium-connector-setup-sysbench.sh` script in `deploy` folder to 
+create the MySQL debezium connector. The connector is setup to read from the `sbtest` table required by SysBench
+
+For sink, the default `sink-connector-setup-schema-registry.sh` script can be executed to create 
+the ClickHouse Sink Connector.
+
+## Insert tests(SysBench)
+`run_sysbench_insert_load_test.sh` script executes the oltp_insert lua script in Sysbench.
+
+## Update/Delete tests(SysBench)
+
+
+## Performance Numbers
+![](img/insert_performance_tests.png) \
+
+`select database, table, event_type, partition_id, count() c, round(avg(rows)) from system.part_log
+where event_date >= today() and event_type = 'NewPart'
+group by database, table, event_type, partition_id
+order by c desc`\
+
+With the SysBench insert tests(6 Kafka partitions), the following are the numbers we observed on 12-core i7 64 GB RAM instance.\
+<b>ClickHouse Rows Insertion Rate: 9k/second, 2.1 MB/second.
+
+## ClickHouse insertion rate
+`select database, table, event_type, partition_id, count() c, round(avg(rows)) from system.part_log
+where event_date >= today() and event_type = 'NewPart'
+group by database, table, event_type, partition_id
+order by c desc`