Update qryn cloud docs

metrico · Aug 30, 2024 · ae4532e · ae4532e
1 parent 7be9be3
commit ae4532e
Show file tree

Hide file tree

Showing 5 changed files with 136 additions and 9 deletions.
diff --git a/docs/cloud/cluster.md b/docs/cloud/cluster.md
@@ -21,14 +21,14 @@ The distribution/sharding is `fingerprint` based to consist session to the same
 1. Create a compose yaml file with two clickhouse servers:
 
 ```yaml
-version: '2.1'
+version: '3.8'
 networks:
   common:
     driver: bridge
 
 services:
   clickhouse:
-    image: clickhouse/clickhouse-server:22.6.4.35
+    image: altinity/clickhouse-server:23.8.11.29.altinitystable
     container_name: clickhouse.min.deploy
     hostname: clickhouse.min.deploy
     domainname: clickhouse.min.deploy
@@ -40,7 +40,7 @@ services:
       - ./config.xml:/etc/clickhouse-server/config.xml
       - ./_clickhouse-data:/var/lib/clickhouse
   clickhouse2:
-    image: clickhouse/clickhouse-server:22.6.4.35
+    image: altinity/clickhouse-server:23.8.11.29.altinitystable
     container_name: clickhouse2.min.deploy
     hostname: clickhouse2.min.deploy
     domainname: clickhouse2.min.deploy
@@ -55,7 +55,7 @@ services:
 
 2. Export the default config.xml file from clickhouse server docker image:
 ```
-docker run -it clickhouse/clickhouse-server:22.6.4.35 cat /etc/clickhouse-server/config.xml >config.xml
+docker run -it clickhouse/clickhouse-server cat /etc/clickhouse-server/config.xml >config.xml
 ```
 
 3. `vi config.xml` or use your preferred text editor

diff --git a/docs/cloud/collector.md b/docs/cloud/collector.md
@@ -44,7 +44,7 @@ docker pull qxip/qryn-cloud-collector
 Define a docker compose file to configure each element.
 
 ```yml
-version: '3.1'
+version: '3.8'
 
 services:
    qryn-collector:

diff --git a/docs/cloud/config.md b/docs/cloud/config.md
@@ -16,10 +16,116 @@ Stack configuration is 100% based on [ENV](env.md) parameters passed to the proc
 
 <!-- tabs:start -->
 
+#### ** qryn-ctrl **
+<a id=ctrl name=ctrl></a>
+
+See an example, where you use ENV vars as defined on the URL above:
+```
+  qryn-ctrl:
+    image: ${version_qryn_ctrl}
+    container_name: qryn-ctrl
+    depends_on:
+      clickhouse-server:
+        condition: service_healthy
+    environment:
+      - QRYN_DATABASE_DATA_0_NAME=${qryn_db_name:-qryn}
+      - QRYN_DATABASE_DATA_0_HOST=${clickhost:-clickhouse-server}
+      - QRYN_DATABASE_DATA_0_PORT=${clickhouse_port:-9000}
+      - QRYN_DATABASE_DATA_0_USER=${clickuser:-default}
+      - QRYN_DATABASE_DATA_0_PASS=${clickpass:-clickPass}
+      - QRYN_DATABASE_DATA_0_DEBUG=${qryn_debug:-true}
+      - QRYN_DATABASE_DATA_0_TTL_DAYS=${qryn_db_ttl:-7}
+      - QRYN_DATABASE_DATA_0_SECURE=${qryn_db_secure:-false}
+      - QRYN_DATABASE_DATA_0_ASYNC_INSERT=${qryn_db_async_insert:-false}
+    command:
+      - sh
+      - -c
+      - ./qryn-ctrl -initialize_db
+```
+
 #### ** qryn-go **
 <a id=api name=api></a>
 
+See an example, where you use ENV vars as defined on the URL above:
+```
+  qryn-go:
+    image: ${version_qryn_reader}
+    container_name: qryn-go
+    restart: unless-stopped
+    volumes:
+      - qryn_reader_data:/var/lib/.prof
+      - qryn_reader_data:/var/lib/.local
+    expose:
+      - ${qryn_go_port}
+    ports:
+      - 3200:${qryn_go_port}
+    environment:
+      - QRYNCLOUD_LICENSE=${qryn_license}
+      - QRYN_SYSTEM_SETTINGS_LICENSE_AUTO_SHUTDOWN=${qryn_license_auto_shutdown:-true}
+      - QRYN_DATABASE_DATA_0_NAME=${qryn_db_name:-qryn}
+      - QRYN_DATABASE_DATA_0_HOST=${clickhost:-clickhouse-server}
+      - QRYN_DATABASE_DATA_0_PORT=${clickhouse_port:-9000}
+      - QRYN_DATABASE_DATA_0_DEBUG=${qryn_debug:-true}
+      - QRYN_DATABASE_DATA_0_USER=${clickuser:-default}
+      - QRYN_DATABASE_DATA_0_PASS=${clickpass:-clickPass}
+      - QRYN_HTTP_SETTINGS_PORT=${qryn_go_port:-3200}
+      - QRYN_HTTP_SETTINGS_DEBUG=${qryn_http_debug:-true}
+      - QRYN_LOG_SETTINGS_STDOUT=${qryn_log_stdout:-true}
+      - QRYN_LOG_SETTINGS_LEVEL=${qryn_log_lvl:-debug}
+    depends_on:
+        clickhouse-server:
+          condition: service_healthy
+        qryn-ctrl:
+          condition: service_completed_successfully
+    logging:
+      driver: "local"
+      options:
+        max-size: "10m"  # Rotate logs when they reach 10MB
+        max-file: "5"    # Keep up to 5 rotated log files
+```
+
 #### ** qryn-writer **
 <a id=writer name=writer></a>
 
+See an example, where you use ENV vars as defined on the URL above:
+```
+  qryn-writer:
+    image: ${version_qryn_writer}
+    container_name: qryn-writer
+    restart: unless-stopped
+    volumes:
+      - qryn_writer_data:/var/lib/.prof
+      - qryn_writer_data:/var/lib/.local
+    expose:
+      - ${qryn_writer_port}
+    ports:
+      - 3100:${qryn_writer_port}
+    environment:
+      - QRYNCLOUD_LICENSE=${qryn_license}
+      - QRYN_SYSTEM_SETTINGS_LICENSE_AUTO_SHUTDOWN=${qryn_license_auto_shutdown:-true}
+      - QRYN_DATABASE_DATA_0_NAME=${qryn_db_name:-qryn}
+      - QRYN_DATABASE_DATA_0_HOST=${clickhost:-clickhouse-server}
+      - QRYN_DATABASE_DATA_0_PORT=${clickhouse_port:-9000}
+      - QRYN_DATABASE_DATA_0_DEBUG=${qryn_debug:-true}
+      - QRYN_DATABASE_DATA_0_USER=${clickuser:-default}
+      - QRYN_DATABASE_DATA_0_PASS=${clickpass:-clickPass}
+      - QRYN_HTTP_SETTINGS_PORT=${qryn_writer_port:-3100}
+      - QRYN_LOG_SETTINGS_STDOUT=${qryn_log_stdout:-true}
+      - QRYN_LOG_SETTINGS_LEVEL=${qryn_log_lvl:-debug}
+    command:
+      - sh
+      - -c
+      - ./cloki-writer
+    depends_on:
+      clickhouse-server:
+        condition: service_healthy
+      qryn-ctrl:
+          condition: service_completed_successfully
+    logging:
+      driver: "local"
+      options:
+        max-size: "10m"  # Rotate logs when they reach 10MB
+        max-file: "5"    # Keep up to 5 rotated log files
+```
+
 <!-- tabs:end -->
diff --git a/docs/cloud/env.md b/docs/cloud/env.md
@@ -20,6 +20,20 @@ The following `ENV` parameters can be used to configure and control **qryn:cloud
 
 <!-- tabs:start -->
 
+#### ** qryn-ctrl **
+<a id=ctrl name=ctrl></a>
+| ENV   	                |Default   	|Usage   	|
+|------------------------|---	    |---		|
+| QRYN_DATABASE_DATA_0_NAME | qryn | qryn DB name |
+| QRYN_DATABASE_DATA_0_HOST | "http://localhost" | Clickhouse Server address |
+| QRYN_DATABASE_DATA_0_PORT | 9000 | Clickhouse Binary Port |
+| QRYN_DATABASE_DATA_0_USER | default | Clickhouse Username |
+| QRYN_DATABASE_DATA_0_PASS | qryn | Clickhouse Password |
+| QRYN_DATABASE_DATA_0_DEBUG | false | Set to true to see debug messages about database in log |
+| QRYN_DATABASE_DATA_0_TTL_DAYS | 0 | Set the data rotation policy - aka for how long to keep data before pruning it. Most be overrided. |
+| QRYN_DATABASE_DATA_0_SECURE | false | Set to true when Clickhouse is hosted on HTTPS (optional) |
+| QRYN_DATABASE_DATA_0_ASYNC_INSERT | false | Set to true if you want to ingest data into the DB asyncronously to data being received |
+
 #### ** qryn-go **
 <a id=api name=api></a>
 

diff --git a/docs/cloud/installation.md b/docs/cloud/installation.md
@@ -8,7 +8,7 @@
 
 !> qryn:cloud requires activation & pull tokens! Please [contact us](mailto:[email protected]) to obtain a license.
 
-?> ClickHouse Server LTS version >=22 and <=22.9 is suggested. _22.10+ not yet supported._
+?> ClickHouse Server LTS version >=23 is suggested.
 
 ## 📦 Installation
 
@@ -52,7 +52,7 @@ Define a docker compose file to configure each element.
 ?> Adjust the clickhouse server and authentication details to match your setup
 
 ```yml
-version: '3.1'
+version: '3.8'
 
 services:
    qryn-ctrl:
@@ -162,6 +162,10 @@ Use `Kubernetes` and `helm` to get started using either a local or cloud ClickHo
 
 ?> You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts.
 
+Follow this [guide](https://github.com/metrico/qryn-cloud-helm) to get started with K8s helm charts.
+
+**Any other steps below show you how to run Qryn in K8s without Helm.**
+
 ##### Pull the qryn service containers
 ```bash
 docker pull qxip/qryn-writer-cloud
@@ -342,7 +346,10 @@ dpkg -i qryn-writer-cloud_vx.x.x_amd64.deb
 dpkg -i qryn-go-cloud_vx.x.x_amd64.deb
 ```
 ##### Configure
+Configure per [here](./config.md)
 ```bash
+vim /etc/qryn-writer/qryn-writer.json
+vim /etc/qryn-go/qryn-go.json
 ```
 ##### Run
 ```bash
@@ -372,7 +379,7 @@ dpkg -i qryn-writer-cloud_vx.x.x_x86_64.rpm
 dpkg -i qryn-go-cloud_vx.x.x_x86_64.rpm
 ```
 ##### Configure
-Configure per [here](/cloud/config.md)
+Configure per [here](./config.md)
 ```bash
 vim /etc/qryn-writer/qryn-writer.json
 vim /etc/qryn-go/qryn-go.json
@@ -414,7 +421,7 @@ The default rotation mechanism is configured via amount of days for each node of
 QRYN_DATABASE_DATA_0_TTL_DAYS = 10
 ```
 
-?> The default days value is "7"
+?> The default days value is "0". It **must** be configured during the setup.
 
 ### Space based rotation