You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Looks like the cache is building up as soon as the query is executed & we get the result.
Is there a way to clear this cache & get rid of this memory build up issue?
We are trying the below thing.
Automatic Cache Reset with Presto Configuration Settings
While Presto doesn't support direct cache invalidation, you can configure some parameters to manage memory and reduce the impact of caching:
Memory Spill and Query Timeouts
If you want Presto to handle memory more dynamically and avoid excessive memory usage, enable spilling and set query timeouts:
Enable Spilling (if not already enabled):
experimental.spill-enabled=true
query.max-run-time=5m
But looks like it is not a sophisticated way.
We are currently re-launching the service whenever there is a high memory usage. Could you suggest a better approach?
The text was updated successfully, but these errors were encountered:
Presto version: 0.278.1
Presto worker nodes memory consumption is gradually increasing over a period of time & eventually its killing the service.
Hi, we are using Presto with the below configs:
One coordinator & 3 workers.
Coordinator configs:
coordinator=true
node-scheduler.include-coordinator=true
http-server.http.port=49152
node-scheduler.max-splits-per-node=2500
query.max-run-time=5m
#task.max-worker-threads=24
query.max-memory=12GB
query.max-memory-per-node=2GB
discovery-server.enabled=true
discovery.uri=http://localhost:49152
join-distribution-type=BROADCAST
optimizer.join-reordering-strategy=NONE
worker configs:
coordinator=false
http-server.http.port=49001
node-scheduler.max-splits-per-node=2500
query.max-run-time=5m
#task.max-worker-threads=64
query.max-memory=12GB
query.max-memory-per-node=2GB
#discovery-server.enabled=true
discovery.uri=http://:49152
Looks like the cache is building up as soon as the query is executed & we get the result.
Is there a way to clear this cache & get rid of this memory build up issue?
We are trying the below thing.
Automatic Cache Reset with Presto Configuration Settings
While Presto doesn't support direct cache invalidation, you can configure some parameters to manage memory and reduce the impact of caching:
Memory Spill and Query Timeouts
If you want Presto to handle memory more dynamically and avoid excessive memory usage, enable spilling and set query timeouts:
Enable Spilling (if not already enabled):
experimental.spill-enabled=true
query.max-run-time=5m
But looks like it is not a sophisticated way.
We are currently re-launching the service whenever there is a high memory usage. Could you suggest a better approach?
The text was updated successfully, but these errors were encountered: