Sbt-assembly is using lots of native memory #517
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue description
On some of our build relying on
sbt-assembly
to create shaded jar we have observed big usage of native memory.For a 6G max heap JVM, the pod memory usage is above 12G
Analysing the native usage with
libtcmalloc_and_profiler.so.4
andgoogle-pprof
we observed that the native usage is due to native usage of Inflater used to unzip jar:Src code ref of the native inflater code: Java_java_util_zip_Inflater_init and Java_java_util_zip_Inflater_inflateBytesBytes
Proposal
Current code keep all jar file handler which leads to all native inflater being kept with all the memory those are using
In order to avoid this, the proposal is to load all entry data from each jar as ByteArrayInputStream instead of keeping the stream and immediately close the jar once all entries are loaded
Here the native memory usage after this change
The counterpart is an increase of the heap usage.
On our specific build we had to increase heap size to 7G but pod memory usage has decreased below 10G: