Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in alloc::raw_vec::finish_grow #954

Open
iameli opened this issue Mar 5, 2025 · 1 comment
Open

Memory leak in alloc::raw_vec::finish_grow #954

iameli opened this issue Mar 5, 2025 · 1 comment

Comments

@iameli
Copy link
Contributor

iameli commented Mar 5, 2025

Hi! I'm tracking down a memory leak in Streamplace, reported as an outstanding allocation of alloc::raw_vec::finish_grow that never gets freed.

Background

Streamplace is a Go program that facilitates livestreaming on Bluesky's AT Protocol. Toward that end, users send in livestreams, those livestreams are segmented into one-second MP4 files, and those MP4 files are signed with a users' key. We make use of C2PA signing for this, using their c2pa-rs Rust library. To get the Rust library in the Go program, we've written c2pa-go, which is itself forked from the UniFFI wrappers in c2pa-python. So the full toolchain is something like this:

streamplace --> c2pa-go --> uniffi-bindgen-go --> UniFFI --> c2pa-rs.

The memory leak could feasibly be happening on any part of that chain, so I'm opening issues on all of those repos to cross-reference.

uniffi-bindgen-go: NordSecurity/uniffi-bindgen-go#67
streamplace: streamplace/streamplace#1
uniffi-rs: mozilla/uniffi-rs#2470

The leak

The leak is easiest to visualize on this jemalloc output PDF, which gives a hint of the c2pa-rs stack trace: profile copy.pdf

Full profile is here: jeprof.64565.0.f.heap.tar.gz

But it can also be seen, absent any stack trace, in this output from sudo memleak-bpfcc -p [streamplace pid] -o 60000 30:

	18710864 bytes in 44311 allocations from stack
		_ZN5alloc7raw_vec11finish_grow17hb0d464f0a70eb9b5E.llvm.15444882852554099438+0x2d [streamplace]

So something is allocating some raw_vec space for the video files and never freeing it. By using Go pprof tooling I can validate that Go doesn't have a reference to that data anywhere, so that shouldn't be a concern.

Still investigating, will update more as I find things out.

@iameli
Copy link
Contributor Author

iameli commented Mar 5, 2025

Also worth pointing out my c2pa-rs fork is branched off of v0.33.3 right now; I'm working on getting the secp256k1 stuff ported over to the latest release and then I'll check again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant