Heed + rkyv Decoding alignment issues

[fabianmurariu]

I'm trying to get heed + lmdn and rkyv to do zero-copy de-serialization but I'm getting
Error: Decoding(ContextError(ArchiveError(Underaligned { expected_align: 8, actual_align: 1 }))) is there a way to get the correct alignment out of heed/lmdb?

this is the code
https://gist.github.com/fabianmurariu/d611da188be44d7b48dedf1b3590bf7d#file-lib-rs-L99

[Kerollmops]

Hey @fabianmurariu 👋

You can read more about the LMDB relocation function in this thread. Unfortunately, it is not possible to force the alignment of the key/value with LMDB.

However, I would suggest you comment on the OpenLDAP ITS issue or create a new one to make it possible in the future. Howard Chu told us that Symas hired a DB developer to work on different issues.

On the other hand, you could use sanakirja to serialize and deserialize rkyv data on a disk without any copy.

[jean-airoldie]

@Kerollmops

Based on this email exchange with Howard Chu as well as this earlier one, it seems you can preserve alignment as long as you pad both keys & values.

LMDB's own headers are aligned to 16 byte boundaries. If your keys+values are also aligned to 16 byte boundaries then nodes shuffling around will have no effect on their alignment.

So in this specific examples, padding both keys and values to force 8 byte alignment should solve this issue.

[Kerollmops]

Hey @jean-airoldie 👋

Thank you for the info. However, I think library users will have to do that themselves. You could help others by adding a padding/alignment section to the lib.rs documentation 😃

[praveenperera]

Could you not use rkyv AlignedVec at the time of serialization?

Actually I think since 0.8 types are automatically aligned at serialization, its opt-out: https://rkyv.org/format/alignment.html

[jean-airoldie]

@praveenperera In case you're talking about me, I don't use rkyv myself but some other zero-copy deserialization format. I just searched the repo for mentions of alignment guarantees and I found this issue.

@Kerollmops I can't right now, and I'm gonna be busy for the next few days, but if i remember I'll add a section in the docs when I can. However I can't promise I will remember.