Performance issues related to Latency observed with Jedisson Bucket4j

[taru-svg]

Hi @vladimir-bukhtoyarov ,

Thanks for providing this library!
I am using Bucket4j with Jedis integration and AWS Elasticache (Redis) and running a Performance test.
With around 80 users running for 10 mins, I see the latency of .tryConsumeAndReturnRemaining(1) is around 4 ms.

But with the increase in users this method (tryConsumeAndReturnRemaining) takes around 300 ms

Here in my configuration, I am using Java 11 and Springboot

@Bean
JedisConnectionFactory jedisConnectionFactory(JedisPoolConfig jedisPoolConfig, RedisClusterConfiguration redisClusterConfiguration) {
  
    RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration(host, port);
    redisStandaloneConfiguration.setPassword(pwd);

    JedisClientConfiguration jedisClientConfiguration = JedisClientConfiguration
            .builder()
            .connectTimeout(Duration.ofSeconds(60)) // Added connect timed out
            .useSsl()
            .and()
            .usePooling()
            .poolConfig(jedisPoolConfig)
             // Todo: Try Pooling enabled, vertically scale the machine, horizontally scale the machine to cluster mode
            .build();



    return new JedisConnectionFactory(redisStandaloneConfiguration , jedisClientConfiguration);
}

@Bean
    public JedisBasedProxyManager<byte[]> jedisBasedProxyManager(JedisPool jedisPool) {
        return JedisBasedProxyManager.builderFor(jedisPool)
                .withExpirationStrategy(ExpirationAfterWriteStrategy.basedOnTimeForRefillingBucketUpToMax(Duration.ofSeconds(3)))
                .build();
   }

@Bean
    JedisPoolConfig jedisPoolConfig() {
        final JedisPoolConfig poolConfig = new JedisPoolConfig();
        poolConfig.setMaxTotal(300);
        poolConfig.setMaxIdle(300);
        return poolConfig;
    }

@Bean(value = "redisTemplate")
    public RedisTemplate<String, Object> redisTemplate(JedisConnectionFactory jedisConnectionFactory) {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(jedisConnectionFactory);
        return redisTemplate;
    }

@Bean
    JedisPool jedisPool(JedisPoolConfig poolConfig) {
        final JedisPoolConfig poolConfig = new JedisPoolConfig();
        poolConfig.setMaxTotal(128);
        poolConfig.setMaxIdle(128);

        return new JedisPool(poolConfig, host, port, 60000, pwd, 0, true);
    }

@Bean(name = "cacheManager")
public CacheManager cacheManager(JedisConnectionFactory jedisConnectionFactory) {
    return RedisCacheManager.builder(jedisConnectionFactory)
            .cacheDefaults(RedisCacheConfiguration.defaultCacheConfig())
            .build();
}

For accessing bucket:

jedisBasedProxyManager.builder().build(bucketKey.getBytes(), bucketConfiguration(tokens))


private BucketConfiguration bucketConfiguration(long tokens) {
        Refill refill = Refill.intervally(tokens, Duration.ofSeconds(1));
        Bandwidth limit = Bandwidth.classic(tokens, refill);
        return BucketConfiguration.builder().addLimit(limit).build();
    }

`

Can you please help me identify, if I am doing something wrong.

Also I had one another question, When I use a distributed cache like Elasticache redis, does it go and update the tokens in that bucket for each request. We have multiple endpoints, hence the bucketKey we're using is userId_endpointPath

So I am testing this for 1 user and one single endpointPath, so does it update the token for each request in the backend path, or it maintains a cache on the machine locally and then updates the distributed cache periodically.

I checked this documentation on Bucket4j and it mentions the Async is not supported for Jedis, could that be a potential reason:

https://bucket4j.com/8.10.1/toc.html#bucket4j-redis

[vladimir-bukhtoyarov]

Hello @taru-svg

So I am testing this for 1 user and one single endpointPath, so does it update the token for each request in the backend path, or it maintains a cache on the machine locally and then updates the distributed cache periodically.

By default, an interaction with bucket leads to immediate request to Redis.

With around 80 users running for 10 mins, I see the latency of .tryConsumeAndReturnRemaining(1) is around 4 ms.
But with the increase in users this method (tryConsumeAndReturnRemaining) takes around 300 ms

Did you use same key for all actors in load test?

P.S. please read these issues that describe current state of Redis perfomanse #404 #479 #407

[taru-svg]

Thanks for attaching the links @vladimir-bukhtoyarov

Yes, I am using the same key (userId_path) for my performance testing

[vladimir-bukhtoyarov]

Yes, I am using the same key (userId_path) for my performance testing

Rewriting the load according to proposed usage scenario (key per user instead of single key for all user) should allow to achive good perfomance. All Bucket4j Redis integrations is built on top of Compare&Swap, that is why all of them is not well scalable(by default if no optimizations are configured) in case of huge contention on single key.

[taru-svg]

We're using different keys for each user, as key consists of userId, which will be different for each user.
But for the sake of performance testing I am testing with one user. In reality each user can make 20K requests/second.

I checked this enhancement feature: #410 , would it be possible to do a POC for this, to check if it optimizes our latency.

[taru-svg]

@vladimir-bukhtoyarov Thanks for helping, it works good with delayed optimization, will be tracking the progress of
this enhancement feature: #410