Skip to content

Add support for multiple ClickHouse configurations #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

ZLBillShaw
Copy link

This PR adds the ability to support multiple ClickHouse configurations in the system. It introduces changes to config parsing and adjusts connection logic accordingly.

@CLAassistant
Copy link

CLAassistant commented May 20, 2025

CLA assistant check
All committers have signed the CLA.

@serprex
Copy link
Member

serprex commented Jun 5, 2025

This PR seems to contain changes outside the scope of its description

Multiconfig seems like a nonstandard convention which we would prefer not to go forward with. Could you describe your use case & the challenge you have adjusting environment variables based on your deployment environment?

@serprex serprex self-requested a review June 5, 2025 17:55
@ZLBillShaw
Copy link
Author

Thanks for the feedback!

Let me clarify the use case first:

In our deployment, we have multiple ClickHouse clusters across different environments and regions (e.g., per-tenant or per-datacenter deployments). We frequently need to query different clusters depending on the context, often in parallel or without restarting the collector.

The existing single-config model forces us to restart or redeploy with updated environment variables every time we want to switch ClickHouse targets, which is operationally inconvenient and not scalable for larger multi-tenant environments.

That's why we proposed a multi-config approach to support multiple configured ClickHouse targets that can be selected dynamically during runtime. This helps us run a single instance of the collector that can handle data routing to the appropriate ClickHouse without restarts or redeployments.

@serprex serprex requested a review from iskakaushik June 6, 2025 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants