Replies: 1 comment
-
Did you check network throughput? Redis/Application CPU load? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
We are facing delays in task execution from redisson when we are scaling to more than 100 schedules per second. Currently once we observe a delay in a schedule it does not recover and we have to reschedule. Can RScheduledExecutorService can support such use case(> 100 schedules) ? If so is there any setting to tune polling the redis queue for tasks , batch size..etc?
Attaching below the code we are currently using.
` // RScheduledExecutorService init
@getter
public class RunnableScheduler implements Runnable , Serializable {
}`
Also we are getting below this error sometimes which may also be the culprit
2023-01-27 05:21:52,940 [] [pool-19-thread-12] ERROR org.redisson.executor.RedissonExecutorRemoteService - Can't execute: RemoteServiceRequest [requestId=7cfae35853efd9fc4bff040942426bb1, methodName=scheduleAtFixedRate, signature=[2954431956196527280, 5708544082777223948], args=[org.redisson.executor.params.ScheduledAtFixedRateParameters@220849aa], options=RemoteInvocationOptions[ackTimeoutInMillis=null, executionTimeoutInMillis=null], date=1674796911580] java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.redisson.executor.RedissonExecutorRemoteService.invokeMethod(RedissonExecutorRemoteService.java:107) at org.redisson.RedissonRemoteService.lambda$executeMethod$11(RedissonRemoteService.java:434) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.redisson.client.RedisResponseTimeoutException: Redis server response timeout (300 ms) occured after 0 retry attempts, is non-idempotent command: false Check connection with Redis node: redishost.com/10.9.0.27:15730 for TCP packet drops. Try to increase nettyThreads and/or timeout settings. Command: (EVAL), params: [redis.call('zrem', KEYS[5], 'ff' .. ARGV[3]);if redis.call('decr', KEYS[1]) == 0 then redis.call('del', KEYS[1]);if redis.call('get', KEYS[2]) == ARGV[1] then redis.call('del', KEYS[6]);redis.call('set', KEYS[2], ARGV[2]);redis.call('publish', KEYS[3], ARGV[2]);end;end;, 6, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:counter, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:status, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:termination-topic, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:tasks, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:scheduler, {GlobalExecutor:org.redisson.executor.RemoteExecutorService}:retry-interval, 1, 2, ...], channel: [id: 0xf1201c74, L:/172.31.17.128:36062 - R:redishost.com/10.9.0.27:15730] at org.redisson.command.RedisExecutor.lambda$scheduleResponseTimeout$8(RedisExecutor.java:387) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more
Any help on this will be much appreciated.
Thanks,
sanre6
Beta Was this translation helpful? Give feedback.
All reactions