Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write book section on wrapper types and add niching example to examples #429

Open
GeeWee opened this issue Sep 27, 2023 · 0 comments
Open
Labels
documentation Improvements or additions to documentation
Milestone

Comments

@GeeWee
Copy link
Contributor

GeeWee commented Sep 27, 2023

Context: We operate in a pretty size-constrained environment and we're using rkyv to embed static data into our application, which works great. Recently I've been spending some time reducing the size of our data payloads. Particularly we have a lot of optional floats, that I'd prefer to optimize the size of.
E.g. see this issue. e.g. storing an Option takes up 16 bytes, but just storing an f64 only takes up 8.

I know all my floats are positive, which means it should be possible for me to use f32::MIN as a sentinel value - essentially I can see if the value is f32::MIN, it's a None, which means I'd be able to skip the size overhead of Option<f32>

I figured I could accomplish this with an ArchiveWith wrapper, but I'm unsure. I wanted to have an Option<f32>/ArchivedOption<f32> as the user-facing type, while just serializing it as an f32.

I'm a bit stumped about whether or not this is possible. I've read the rkyv book, the ArchiveWith trait, but I'm a bit stumped, as the documentation for use-cases like this isn't great.

This is as far as I've gotten, but it has several issues.

  • The Archived type is f32 where I'd like it to be Option<f32> for both the archived and non-archived variant
  • I perform the same logic in deserialize and resolve_with, and I'm not sure how to avoid it. I assume my type bounds are wrong somewhere.
pub struct OptionF32SentinelValue;

impl ArchiveWith<Option<f32>> for OptionF32SentinelValue {
    type Archived = f32;
    type Resolver = ();

    unsafe fn resolve_with(
        field: &Option<f32>,
        pos: usize,
        resolver: Self::Resolver,
        out: *mut Self::Archived,
    ) {
        // Coerce to sentinel value for None
        match field {
            None => f32::MIN.resolve(pos, (), out),
            Some(float) => float.resolve(pos, (), out),
        }
    }
}

impl<S: Fallible + ?Sized> SerializeWith<Option<f32>, S> for OptionF32SentinelValue
where
    Option<f32>: Serialize<S>,
{
    fn serialize_with(field: &Option<f32>, serializer: &mut S) -> Result<Self::Resolver, S::Error> {
        // Coerce to sentinel values for None
        let value_to_serialize = match field {
            None => f32::MIN,
            Some(f) => *f,
        };
        let result = rkyv::Serialize::serialize(&value_to_serialize, serializer)?;

        Ok(result)
    }
}

impl<D: Fallible + ?Sized> DeserializeWith<Archived<f32>, Option<f32>, D> for OptionF32SentinelValue
where
    Archived<f32>: Deserialize<f32, D>,
{
    fn deserialize_with(
        field: &Archived<f32>,
        deserializer: &mut D,
    ) -> Result<Option<f32>, D::Error> {
        let res = field.deserialize(deserializer)?;

        // Convert sentinel values back to None
        if res == f32::MIN {
            return Ok(None);
        }

        Ok(Some(res))
    }
}

Questions

I've been stumped by the following questions during the process:

  • Do you implement ArchiveWith for the type that you want archived, e.g. in my case Option<f32>?
  • What is Resolver, when is one needed? I think it's just metadata based on this page, but I can see the ArchiveWith example e.g. uses Resolver<i32> which I can't quite figure out what does, or what metadata it keeps. As far as I can tell a fair amount of rkyv types just use () as the resolver
  • It's very easy to get the deserialize implementation wrong with the type bounds. That means you get an error like this when trying to deserialize:
let deserialized: Example = archived.deserialize(&mut Infallible).unwrap();
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `Example`, found `With<_, _>` 

which isn't very easy to determine what actually means. An explanation of how the types in ArchiveWith and DeserializeWith should align would be extremely helpful

I'd be happy to contribute back to the docs with the answers to these questions

@djkoloski djkoloski changed the title Documentation for wrappers that change the type of the Archived Representation Write book section on wrapper types and add niching example to examples Mar 9, 2024
@djkoloski djkoloski added this to the v0.8 milestone Mar 9, 2024
@djkoloski djkoloski added the documentation Improvements or additions to documentation label Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants