Direct (non heap) memory leaks caused by serving bigger files (like videos)

[kajism]

When a file is bigger than the org.http-kit.memmap-file-threshold (20MB by default) then MappedByteBuffer is used, which is not so easy to GC and http-kit seems not to be trying to do so.

See https://stackoverflow.com/questions/25238110/how-to-properly-close-mappedbytebuffer .

This memory can, by Java default, grow up to the Heap size limit which means your app memory consumption can easily exceed twice the set maximum heap size and then serving large files stops.
Direct buffers memmory can be watched using Mbeans (java.nio:name=direct,type=BufferPool MemoryUsed) in jconsole.

As noted in org.http-kit.HttpUtils.java source in comment on line 111:
// serving file is better be done by Nginx

Or in this scenario, Undertow can be recommended.

[ptaoussanis]

@kajism Thanks for creating this 👍

As noted in org.http-kit.HttpUtils.java source in comment on line 111:
// serving file is better be done by Nginx

Correct, http-kit is in several areas not well optimised for large payloads.
For this and other reasons (incl. SSL termination and security), the general advice is to use http-kit behind a proxy like nginx.

Part of http-kit's original design ethos (as I understood it) was to try keep the software simple by intentionally keeping its objectives relatively limited. It's unlikely an application server is going to be competitive with a dedicated proxy for tasks like static file serving, for example. So instead of trying to compete, encourage folks to use software optimised for the job (e.g. nginx for static files and SSL termination, and http-kit for dynamic application-level responses).

There's of course some tradeoffs to this, and it won't be the right choice for everyone.

Also, it really would be good to have this advice directly in the README (PRs welcome!).

Otherwise, PRs also always welcome to help improve support for larger payloads.
A recent relevant example includes #479

[kajism]

I have tried to create PR adding another JVM property which would enable/disable reading files using MappedByteBuffer in conjunction with the JVM property added in #479. Unfortunately the allocated direct memory grows anyway, so the problem seems to be deeper. Because I'm currently happy with Undertow, I give up.

[foromer4]

Having the same issue,
getting ByeBuffer from S3 and passing them as is to http-kit response, cause a memory leak in direct memory.
Is there a way to use heap memory instead of direct memory?
Since not based on netty can't find documentation for any flags to achieve
this?