Memory limit exceeded

[ShivaniSK]

Hi @alex-karpovich
We notice below warnings in timebase server logs.
Does it indicate low timebase server memory?
Or do we need to tweak this property DTimeBase.qql.maxGroupsCount=1000000 ?

Can this impact the timebase query performance?

15 Apr 12:39:51 INFO Cache 'GroupsCache' created in EhcacheManager.
15 Apr 12:39:52 INFO Cache 'GroupsCache' removed from EhcacheManager.
15 Apr 13:03:58 WARN QQL groups count is limited by 300000. Cause: memory limit exceeded (free memory: 1036Mb; Memory grow speed: 959Mb per 50000 groups)
15 Apr 13:07:42 WARN QQL groups count is limited by 250000. Cause: memory limit exceeded (free memory: 705Mb; Memory grow speed: 844Mb per 50000 groups)
15 Apr 13:07:43 WARN QQL groups count is limited by 100000. Cause: memory limit exceeded (free memory: 1301Mb; Memory grow speed: 875Mb per 50000 groups)
15 Apr 13:15:57 WARN QQL groups count is limited by 50000. Cause: memory limit exceeded (free memory: 154Mb; Memory grow speed: 731Mb per 50000 groups)
15 Apr 13:17:15 WARN QQL groups count is limited by 50000. Cause: memory limit exceeded (free memory: 411Mb; Memory grow speed: 789Mb per 50000 groups)

Regards,
Shivani

[alex-karpovich]

Greetings,
This mean you used GROUP BY attribute that have many values. I suspect that you group by timestamp?
Please share your query.

Anyway changing DTimeBase.qql.maxGroupsCount=1000000 won't help if you have 1 million groups for example
Lets investigate first

[ShivaniSK]

Hello @alex-karpovich ,
Yes we are grouping by timestamp. Below is the query

select AVG{} (price) as priceAgg from reverse(market_data) 
where timestamp BETWEEN 1709510400000 AND 1709596799999 
AND SYMBOL IN ('ABC') 
GROUP BY timestamp

the total number of records for above query within timebase are 3,89,780 and after group by it comes around 3,53,713.

There is another query where we are doing roll up over 1m. The total number of records between the provided timerange for 24hrs is around 2Million. After roll up the count comes around 1440.

select LAST{}(bid)
            from obd RESET over TIME (1m)  
            where timestamp BETWEEN 1709510400000 
AND 1709596799899 AND SYMBOL IN ('ABC') 
AND level == '1'

Also wanted to understand if the above warning impacts the overall query performance? If yes, then what steps can we do to improve it.

Regards,
Shivani

[alex-karpovich]

I would suggest to avoid group by timestamp in all scenarios because it's will impact query performance

in your case you can use OVER TIME(1ms)

select AVG{} (price) as priceAgg from reverse(market_data) 
where timestamp BETWEEN 1709510400000 AND 1709596799999 
OVER TIME(1ms)

[ShivaniSK]

Thanks @alex-karpovich , will try this out.
However we also have queries where we GROUP BY on multiple columns including timestamp.
For example

select SUM{}(notional) as notionalAgg 
from reverse(sample_data) 
where timestamp BETWEEN 1685664000000 AND 1685750399999 AND SYMBOL IN ('EURUSD') 
GROUP BY timestamp, side

As suggested by you, we can replace the above query with below

select SUM{}(notional) as notionalAgg 
from sample_data OVER TIME(1ms)
where timestamp BETWEEN 1685664000000 AND 1685750399999 AND SYMBOL IN ('EURUSD') 
GROUP BY side

However, the above query will bring us to the other open issue which we are currently facing when using OVER and GROUP BY together #275

[alex-karpovich]

issue is fixed, please try with latest release https://github.com/finos/TimeBase-CE/releases/tag/6.1.24