Memory usage with posting cache

[RJKeevil]

Continuing the discussion for #9526 @matthewmcneely @xqqp . Please find below my heap output. This is at the exact same point on the same data, the only difference is that one has --cache "size-mb=5120; percentage=40,40,20; remove-on-update=true" and the other has --cache "size-mb=5120; percentage=0,65,35; remove-on-update=true". Memory usage of the former was around 9G at the point of the dump, the latter has around 2G.

heap.zip

I understand that there are many other variables in mem usage, but given that everything else is completely identical, and the size of the memory discrepancy, in this case it does infer that the posting cache is responsible here?

sh-5.2# dgraph version

Dgraph version : 2ec41c9
Dgraph codename : dgraph
Dgraph SHA-256 : 69f19385f47bad662ae92214c67be9177ed3be785e940fa07c3a403f8c9d47d5
Commit SHA-1 : 2ec41c9
Commit timestamp : 2025-12-02 09:52:52 +0100
Branch : v25
Go version : go1.25.4
jemalloc enabled : true

The commit hash there is from my own fork as i periodically bump go.mod for security reasons.

You can see however that this commit is directly on top of 68c24d so that is my effective version.

Thanks for looking into this and shout if you need any more info at all!

[xqqp]

Nothing stands out to me in the heap profile. The posting cache seems to use close to the configured 2 GiB of memory.

In my testing when increasing the cache size (of all three caches) dgraph also used more memory outside of the caches. For example when increasing size-mb=2048 to size-mb=4096, memory usage grew from 5.4 GiB to 8.4 GiB. So 1 GiB higher than expected. I think that is what you are also observing.

I suggest experimenting with the cache size to find a value which gives you good performance at a reasonable memory footprint.

[RJKeevil]

Ok ill give that a try. The counterintuitive thing here is that the cache size-mb is the same in both examples (5120), just the assigned ratio changes (40,40,20 vs 0,65,35). So I would expect the memory usage to be roughly the same, and definitely not 5x larger?

[xqqp]

Ok ill give that a try. The counterintuitive thing here is that the cache size-mb is the same in both examples (5120), just the assigned ratio changes (40,40,20 vs 0,65,35). So I would expect the memory usage to be roughly the same, and definitely not 5x larger?

From the heap profiles you shared, heap_no_posting shows a heap size of 259 MiB and heap_with_posting shows 3809 MiB. Based on that, I think the other two caches are not filled in your use-case, if the posting cache is not used.

[RJKeevil]

Yes you're right, if i set size-mb to 1024 instead it peaks at around 4G usage. So it looks like using 0,65,35 is essentially not using any significant cache space, posting size is the one that matters here, thanks.

I found in the docs the other 2 caches are PstoreBlockCache and PstoreIndexCache, but not much detail on what they are. Hopefully it's not a problem that they have little usage? The workload is around 2 million predicates, mixed read and write.