You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi! I'm tracking down a memory leak in Streamplace, reported as an outstanding allocation of alloc::raw_vec::finish_grow that never gets freed.
Background
Streamplace is a Go program that facilitates livestreaming on Bluesky's AT Protocol. Toward that end, users send in livestreams, those livestreams are segmented into one-second MP4 files, and those MP4 files are signed with a users' key. We make use of C2PA signing for this, using their c2pa-rs Rust library. To get the Rust library in the Go program, we've written c2pa-go, which is itself forked from the UniFFI wrappers in c2pa-python. So the full toolchain is something like this:
But it can also be seen, absent any stack trace, in this output from sudo memleak-bpfcc -p [streamplace pid] -o 60000 30:
18710864 bytes in 44311 allocations from stack
_ZN5alloc7raw_vec11finish_grow17hb0d464f0a70eb9b5E.llvm.15444882852554099438+0x2d [streamplace]
So something is allocating some raw_vec space for the video files and never freeing it. By using Go pprof tooling I can validate that Go doesn't have a reference to that data anywhere, so that shouldn't be a concern.
Still investigating, will update more as I find things out.
The text was updated successfully, but these errors were encountered:
Also worth pointing out my c2pa-rs fork is branched off of v0.33.3 right now; I'm working on getting the secp256k1 stuff ported over to the latest release and then I'll check again.
Hi! I'm tracking down a memory leak in Streamplace, reported as an outstanding allocation of
alloc::raw_vec::finish_grow
that never gets freed.Background
Streamplace is a Go program that facilitates livestreaming on Bluesky's AT Protocol. Toward that end, users send in livestreams, those livestreams are segmented into one-second MP4 files, and those MP4 files are signed with a users' key. We make use of C2PA signing for this, using their
c2pa-rs
Rust library. To get the Rust library in the Go program, we've writtenc2pa-go
, which is itself forked from the UniFFI wrappers inc2pa-python
. So the full toolchain is something like this:streamplace
-->c2pa-go
-->uniffi-bindgen-go
-->UniFFI
-->c2pa-rs
.The memory leak could feasibly be happening on any part of that chain, so I'm opening issues on all of those repos to cross-reference.
uniffi-bindgen-go: NordSecurity/uniffi-bindgen-go#67
streamplace: streamplace/streamplace#1
uniffi-rs: mozilla/uniffi-rs#2470
The leak
The leak is easiest to visualize on this jemalloc output PDF, which gives a hint of the c2pa-rs stack trace: profile copy.pdf
Full profile is here: jeprof.64565.0.f.heap.tar.gz
But it can also be seen, absent any stack trace, in this output from
sudo memleak-bpfcc -p [streamplace pid] -o 60000 30
:So something is allocating some
raw_vec
space for the video files and never freeing it. By using Go pprof tooling I can validate that Go doesn't have a reference to that data anywhere, so that shouldn't be a concern.Still investigating, will update more as I find things out.
The text was updated successfully, but these errors were encountered: