You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context: We operate in a pretty size-constrained environment and we're using rkyv to embed static data into our application, which works great. Recently I've been spending some time reducing the size of our data payloads. Particularly we have a lot of optional floats, that I'd prefer to optimize the size of.
E.g. see this issue. e.g. storing an Option takes up 16 bytes, but just storing an f64 only takes up 8.
I know all my floats are positive, which means it should be possible for me to use f32::MIN as a sentinel value - essentially I can see if the value is f32::MIN, it's a None, which means I'd be able to skip the size overhead of Option<f32>
I figured I could accomplish this with an ArchiveWith wrapper, but I'm unsure. I wanted to have an Option<f32>/ArchivedOption<f32> as the user-facing type, while just serializing it as an f32.
I'm a bit stumped about whether or not this is possible. I've read the rkyv book, the ArchiveWith trait, but I'm a bit stumped, as the documentation for use-cases like this isn't great.
This is as far as I've gotten, but it has several issues.
The Archived type is f32 where I'd like it to be Option<f32> for both the archived and non-archived variant
I perform the same logic in deserialize and resolve_with, and I'm not sure how to avoid it. I assume my type bounds are wrong somewhere.
pubstructOptionF32SentinelValue;implArchiveWith<Option<f32>>forOptionF32SentinelValue{typeArchived = f32;typeResolver = ();unsafefnresolve_with(field:&Option<f32>,pos:usize,resolver:Self::Resolver,out:*mutSelf::Archived,){// Coerce to sentinel value for Nonematch field {None => f32::MIN.resolve(pos,(), out),Some(float) => float.resolve(pos,(), out),}}}impl<S:Fallible + ?Sized>SerializeWith<Option<f32>,S>forOptionF32SentinelValuewhereOption<f32>:Serialize<S>,{fnserialize_with(field:&Option<f32>,serializer:&mutS) -> Result<Self::Resolver,S::Error>{// Coerce to sentinel values for Nonelet value_to_serialize = match field {None => f32::MIN,Some(f) => *f,};let result = rkyv::Serialize::serialize(&value_to_serialize, serializer)?;Ok(result)}}impl<D:Fallible + ?Sized>DeserializeWith<Archived<f32>,Option<f32>,D>forOptionF32SentinelValuewhereArchived<f32>:Deserialize<f32,D>,{fndeserialize_with(field:&Archived<f32>,deserializer:&mutD,) -> Result<Option<f32>,D::Error>{let res = field.deserialize(deserializer)?;// Convert sentinel values back to Noneif res == f32::MIN{returnOk(None);}Ok(Some(res))}}
Questions
I've been stumped by the following questions during the process:
Do you implement ArchiveWith for the type that you want archived, e.g. in my case Option<f32>?
What is Resolver, when is one needed? I think it's just metadata based on this page, but I can see the ArchiveWith example e.g. uses Resolver<i32> which I can't quite figure out what does, or what metadata it keeps. As far as I can tell a fair amount of rkyv types just use () as the resolver
It's very easy to get the deserialize implementation wrong with the type bounds. That means you get an error like this when trying to deserialize:
let deserialized: Example = archived.deserialize(&mut Infallible).unwrap();
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `Example`, found `With<_, _>`
which isn't very easy to determine what actually means. An explanation of how the types in ArchiveWith and DeserializeWith should align would be extremely helpful
I'd be happy to contribute back to the docs with the answers to these questions
The text was updated successfully, but these errors were encountered:
djkoloski
changed the title
Documentation for wrappers that change the type of the Archived Representation
Write book section on wrapper types and add niching example to examples
Mar 9, 2024
Context: We operate in a pretty size-constrained environment and we're using
rkyv
to embed static data into our application, which works great. Recently I've been spending some time reducing the size of our data payloads. Particularly we have a lot of optional floats, that I'd prefer to optimize the size of.E.g. see this issue. e.g. storing an Option takes up 16 bytes, but just storing an f64 only takes up 8.
I know all my floats are positive, which means it should be possible for me to use
f32::MIN
as a sentinel value - essentially I can see if the value isf32::MIN
, it's aNone
, which means I'd be able to skip the size overhead ofOption<f32>
I figured I could accomplish this with an
ArchiveWith
wrapper, but I'm unsure. I wanted to have anOption<f32>/ArchivedOption<f32>
as the user-facing type, while just serializing it as an f32.I'm a bit stumped about whether or not this is possible. I've read the rkyv book, the ArchiveWith trait, but I'm a bit stumped, as the documentation for use-cases like this isn't great.
This is as far as I've gotten, but it has several issues.
f32
where I'd like it to beOption<f32>
for both the archived and non-archived variantdeserialize
andresolve_with
, and I'm not sure how to avoid it. I assume my type bounds are wrong somewhere.Questions
I've been stumped by the following questions during the process:
ArchiveWith
for the type that you want archived, e.g. in my caseOption<f32>
?Resolver<i32>
which I can't quite figure out what does, or what metadata it keeps. As far as I can tell a fair amount ofrkyv
types just use()
as the resolverwhich isn't very easy to determine what actually means. An explanation of how the types in
ArchiveWith
andDeserializeWith
should align would be extremely helpfulI'd be happy to contribute back to the docs with the answers to these questions
The text was updated successfully, but these errors were encountered: