Got NativeDbErr(IncorrectInputData) when remove or update

[tklam]

Hi, thanks for implementing this powerful db in Rust. I have got a model:

#[derive(Serialize, Deserialize, Debug)]
#[native_model(id = 1, version = 1)]
#[native_db]
pub struct Model {
  #[primary_key]
  pub name: String,
  pub networks: Vec<Network>,
}

The struct Network is defined in this way:

#[derive(Serialize, Deserialize, Debug)]
pub struct Network {
    name: String,
    config: Rc<Config>, // Config is a struct containing many String members
    inputs: Vec<u32>,
    optional_input: Option<u32>,
    input_to_value: HashMap<u32, u32>,
    registry: Rc<RefCell<Registry>>,
    cache: Arc<RwLock<Cache>>,  // Cache is a struct with many HashSet and HashMap members
}

#[derive(Serialize, Deserialize, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Beh {
    members: Vec<u32>,
    beh_value: u64,
}

#[derive(Serialize, Deserialize, Debug, Default)]
pub struct Registry {
    pair_to_beh: HashMap<(u32, u32), Beh>,
}

This model could be saved into a file without any problem. But when I read back the model and tried to remove(), update(), or upsert(), native_db threw an error IncorrectInputData. It seems the error was caused by the check current_item == item.value in concrete_remove() in transaction/internal/rw_transaction.rs (line 171):

161     pub(crate) fn concrete_remove(
162         &self,
163         model: Model,
164         item: Input,
165     ) -> Result<(WatcherRequest, Output)> {
166         let keys = &item.secondary_keys;
167         {
168             let mut table: redb::Table<Key, &[u8]> = self.get_primary_table(&model)?;
169             let result = if let Some(current_item) = table.remove(&item.primary_key)? {
170                 let current_item = current_item.value();
171                 if current_item == item.value {
172                     Ok(())
173                 } else {
174                     Err(Error::IncorrectInputData {
175                         value: current_item.to_vec(),
176                     })
177                 }
178             } else {
179                 Err(Error::KeyNotFound {
180                     key: item.primary_key.as_slice().to_vec(),
181                 })
182             };
183             if let Err(Error::IncorrectInputData { ref value }) = result {
184                 table.insert(&item.primary_key, value.as_slice())?;
185             }
186             result?;
187         }

So my questions are:

• Did I use the library correctly? I am very new to Rust and native_db
• Is it the HashSet and HashMap that made current_item == item.value false?
• Can this error be ignored by skipping the check?
• Do you recommend using native_db to store this kind of data structures?

Thank you so much

[vincent-herlemont]

@tklam Thank you for your message. I haven't had the time to look into it yet (I'll try during the week), but at first glance, the serialization of HashSet and HashMap might pose a problem because the order is not guaranteed.

You can run a test with native_model by simply trying to encode and decode your structure. And maybe try using a different serializer than the default one. And maybe use ordermap.

[tklam]

Thanks a lot. I tried encoding and decoding the above structs. native_db worked out of the box using the default serializer. There was no issue at all with HashSet and HashMap. Perhaps my setup was wrong; or it might because the structs listed here are in fact the simplified version of the actual structs in my project that they cannot reflect the real situation. Let me check it thoroughly.

[jokeyrhyme]

I've been experimenting with a similar issue, and for me, it looks like removing the HashMap fields from my model/record type fixes it

Beyond the primitive types, my model/record type has Vec and Option fields, and those seem to work just fine

But as soon as I add the HashMap field, I can't do upset(), etc