Using RMapCache with eviction within Netty channel handler, Netty's NioEventLoop starting to accumulate ScheduledFutureTask, but Redisson's EvictonScheduler does not seem to comprise MapCacheEvictionTask, who's responsibility to clean those items?

[abregar]

Hi,
we are using Redisson in combination with Netty's channel handler. Recently we have upgraded application to use RMapCache.
And since Redis itself does not offer ttl on hset, we decided to use eviction mechanism by Redisson. Starting with established Redisson version 3.17.0 following code was used:
RMapCache<String, String> deviceIds = getMapCache(messageId, profileId); deviceIds.put( deviceId.toString(), "", config.getRedis().getMsgExpiration().getQuantity(), config.getRedis().getMsgExpiration().getUnit());

But under some performance evaluation it was shown that Redisson's EvictionScheduler is accumulating MapCacheEvictionTask toward OutOfMemory ex.

Upgrading Redisson to version 3.23.1 as suggested in #5158
we have added removeEmptyEvictionTask option as i.e. following code:
MapCacheOptions<String, String> options = MapCacheOptions.defaults(); options.removeEmptyEvictionTask(); return redisson.getMapCache(MAP_CACHE_PREFIX + messageId + "_" + profileId, options);

Now EvictionScheduler seems to evict and remove the tasks, but leak moved to Netty's NioEventLoop which is accumulating large amount of executors, as in following snippet from MemoryAnalyzer tool, dump was taken upon oom:

while single(ton) Redisson object seems to not comprise any of those tasks:

The question to start discussion here is whether anyone encountered same or similar case and is willing to share hints where to look further.

Thank you in advance

[mrniko]

how many unique RMapCache objects do you use?

[abregar]

The pattern used is to create unique RMapCache per incoming user, so there might be large amount of objects.

[mrniko]

You can try to decrease https://github.com/redisson/redisson/wiki/2.-Configuration#maxcleanupdelay setting to run cleanup process more frequently.

[abregar]

Ty for the suggestion, already came across this config, but kind-a didn't work out. Even with decreasing the config, Netty's nioeventloop still went to somekind of saturation, stopped responding and oom. Worth mentioning that I came a little late to realization that Redisson itself is using Netty for communication toward Redis and not solely our application. This is important as I have observed that large number of RMapCache objects results in multiple helper structures in Redis. And when Redisson's eviction mechanism is added, amount of stuff in Redis also increase significantly.

Noting here as an uneducated Redisson behavior observation - with some synthetic performance evaluation it was shown that ~200k unique RMapCache objects (comprising small amount of values) and more than one simultaneously running app instance (multiple unique Redisson clients instantiated) brings it to the state above.

Consulting the Redisson docs, which say 'It's recommended to use single instance of RMapCache object with the same name for each Redisson client instance' we have decided to redesign the usage toward small amount of unique instances or maybe even use RSet instead.