Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lift concurrent query semaphore limit by default #25627

Open
hiltontj opened this issue Dec 6, 2024 · 0 comments
Open

Lift concurrent query semaphore limit by default #25627

hiltontj opened this issue Dec 6, 2024 · 0 comments
Assignees
Labels

Comments

@hiltontj
Copy link
Contributor

hiltontj commented Dec 6, 2024

Problem statement

The query executor has a hard-coded limit on the concurrent query semaphore of 10:

concurrent_query_limit: 10,

This should be higher by default, and also be configurable.

Proposed solution

Completely remove the limit by default by setting the default to the max for tokio's semaphore (see Semaphore::MAX_PERMITS).

Make the value configurable in the CLI arguments with a --max-concurrent-queries argument.

Alternatives

Don't allow a configurable limit, and just set it internally to Semaphore::MAX_PERMITS.

Additional context

The value of 10 likely came from IOx, where the default value for that semaphore in the querier is chosen as 10. There, however, that is being set on a per-querier basis, so they can support more concurrent queries in practice via multiple nodes.

See also: #25615

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant