Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document alignment requirements for accessing byte slices #399

Open
w-flo opened this issue Jun 27, 2023 · 2 comments
Open

Document alignment requirements for accessing byte slices #399

w-flo opened this issue Jun 27, 2023 · 2 comments
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@w-flo
Copy link

w-flo commented Jun 27, 2023

Thank you for rkyv, it's a great tool!

I have stumpled upon this comment: #382 (comment)

An alignment of 4-16 is usually required, which you can get with rkyv's AlignedVec container. This alignment invariant is a safety condition of archived_root and is checked by check_archived_root.

Looking at the docs for archived_root, it only lists "byte slice must represent an archived object" and "root object must be stored at the end of the slice" as safety preconditions. So I thought that using rkyv::to_bytes to serialize something, then storing the returned bytes in a file, then later reading the bytes from the file as a Vec<u8> (shouldn't be an issue when mmaping the file, as that is something like 4096-byte aligned) and passing them into archived_root would be okay. However, now I think that's unsound, because the Vec might not be 16-byte aligned.

So I do wonder what exactly the alignment requirements are in this situation. I think adding a bullet point to the archived_root docs might be useful? Is it sufficient to use AlignedVec, or are there cases where the alignment needs to be even larger than 16? Maybe the same question arises for the from_bytes_unchecked function.

For context, my application caches some data (supplied by myself) using rkyv, so it's not at risk of "serialized rkyv data injection", so I don't use the validation feature. I'd still like to use the rkyv API correctly under the "I can trust that the bytes in the file are from rkyv::to_bytes, using the same rkyv struct definitions" assumption.

@djkoloski djkoloski added the documentation Improvements or additions to documentation label Jun 28, 2023
@djkoloski
Copy link
Collaborator

I don't know of any more highly-aligned types off the top of my head, but it's possible that types in the standard library will have a higher alignment than 16.

@w-flo
Copy link
Author

w-flo commented Jun 28, 2023

Thanks for the quick response!

And now that I think about it, I guess it makes sense: The byte slice must have the same alignment as (or a multiple of) the #[derive(Archive)] struct (but a minimum of 4)? Otherwise, it would probably be impossible to implement zero-copy deserialization, as struct members could easily end up where they are not allowed to end up.

If so, I believe that would be pretty useful to have in the archived_root etc. docs as a safety requirement.

@djkoloski djkoloski changed the title Clarify / document alignment requirement for archived_root() Document alignment requirements for accessing byte slices Mar 10, 2024
@djkoloski djkoloski added this to the v0.8 milestone Mar 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants