Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add deserialize helper functions #406

Merged
merged 1 commit into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 17 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,50 +45,45 @@ fn example() -> Result<(), Error> {
}
```

Note if you want to deserialize a column that may have invalid types (i.e. a float where some values may be strings), you can use Serde's `deserialize_with` field attribute:
Calamine provides helper functions to deal with invalid type values. For instance if you
want to deserialize a column which should contain floats but may also contain invalid values
(i.e. strings), you can use the [`deserialize_as_f64_or_none`] helper function with Serde's
[`deserialize_with`](https://serde.rs/field-attrs.html) field attribute:

```rust
use serde::{DataType, Deserialize};
use calamine::{RangeDeserializerBuilder, Reader, Xlsx};
use calamine::{deserialize_as_f64_or_none, open_workbook, RangeDeserializerBuilder, Reader, Xlsx};
use serde::Deserialize;

#[derive(Deserialize)]
struct Record {
metric: String,
#[serde(deserialize_with = "de_opt_f64")]
#[serde(deserialize_with = "deserialize_as_f64_or_none")]
value: Option<f64>,
}

// Convert value cell to Some(f64) if float or int, else None
fn de_opt_f64<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error>
where
D: serde::Deserializer<'de>,
{
let data = calamine::Data::deserialize(deserializer)?;
if let Some(float) = data.as_f64() {
Ok(Some(float))
} else {
Ok(None)
}
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
fn main() -> Result<(), Box<dyn std::error::Error>> {
let path = format!("{}/tests/excel.xlsx", env!("CARGO_MANIFEST_DIR"));
let mut excel: Xlsx<_> = open_workbook(path)?;

let range = excel
.worksheet_range("Sheet1")
.ok_or(calamine::Error::Msg("Cannot find Sheet1"))??;
.worksheet_range("Sheet1")
.map_err(|_| calamine::Error::Msg("Cannot find Sheet1"))?;

let iter_result =
let iter_records =
RangeDeserializerBuilder::with_headers(&["metric", "value"]).from_range(&range)?;

for result in iter_results {
for result in iter_records {
let record: Record = result?;
println!("metric={:?}, value={:?}", record.metric, record.value);
}

Ok(())
}
```

The [`deserialize_as_f64_or_none`] function will discard all invalid values, if you want to
return them as `String` you can use the [`deserialize_as_f64_or_string`] function instead.

### Reader: Simple

```rust
Expand Down
198 changes: 197 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ mod de;
mod errors;
pub mod vba;

use serde::de::DeserializeOwned;
use serde::de::{Deserialize, DeserializeOwned, Deserializer};
use std::borrow::Cow;
use std::cmp::{max, min};
use std::fmt;
Expand Down Expand Up @@ -890,3 +890,199 @@ impl<T> Table<T> {
&self.data
}
}

/// A helper function to deserialize cell values as `i64`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_i64`] method to the cell value, and returns
/// `Ok(Some(value_as_i64))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
pub fn deserialize_as_i64_or_none<'de, D>(deserializer: D) -> Result<Option<i64>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_i64())
}

/// A helper function to deserialize cell values as `i64`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_i64`] method to the cell value, and returns
/// `Ok(Ok(value_as_i64))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
pub fn deserialize_as_i64_or_string<'de, D>(
deserializer: D,
) -> Result<Result<i64, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_i64().ok_or_else(|| data.to_string()))
}

/// A helper function to deserialize cell values as `f64`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_f64`] method to the cell value, and returns
/// `Ok(Some(value_as_f64))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
pub fn deserialize_as_f64_or_none<'de, D>(deserializer: D) -> Result<Option<f64>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_f64())
}

/// A helper function to deserialize cell values as `f64`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_f64`] method to the cell value, and returns
/// `Ok(Ok(value_as_f64))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
pub fn deserialize_as_f64_or_string<'de, D>(
deserializer: D,
) -> Result<Result<f64, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_f64().ok_or_else(|| data.to_string()))
}

/// A helper function to deserialize cell values as `chrono::NaiveDate`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_date`] method to the cell value, and returns
/// `Ok(Some(value_as_date))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_date_or_none<'de, D>(
deserializer: D,
) -> Result<Option<chrono::NaiveDate>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_date())
}

/// A helper function to deserialize cell values as `chrono::NaiveDate`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_date`] method to the cell value, and returns
/// `Ok(Ok(value_as_date))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_date_or_string<'de, D>(
deserializer: D,
) -> Result<Result<chrono::NaiveDate, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_date().ok_or_else(|| data.to_string()))
}

/// A helper function to deserialize cell values as `chrono::NaiveTime`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_time`] method to the cell value, and returns
/// `Ok(Some(value_as_time))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_time_or_none<'de, D>(
deserializer: D,
) -> Result<Option<chrono::NaiveTime>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_time())
}

/// A helper function to deserialize cell values as `chrono::NaiveTime`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_time`] method to the cell value, and returns
/// `Ok(Ok(value_as_time))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_time_or_string<'de, D>(
deserializer: D,
) -> Result<Result<chrono::NaiveTime, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_time().ok_or_else(|| data.to_string()))
}

/// A helper function to deserialize cell values as `chrono::Duration`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_duration`] method to the cell value, and returns
/// `Ok(Some(value_as_duration))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_duration_or_none<'de, D>(
deserializer: D,
) -> Result<Option<chrono::Duration>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_duration())
}

/// A helper function to deserialize cell values as `chrono::Duration`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_duration`] method to the cell value, and returns
/// `Ok(Ok(value_as_duration))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_duration_or_string<'de, D>(
deserializer: D,
) -> Result<Result<chrono::Duration, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_duration().ok_or_else(|| data.to_string()))
}

/// A helper function to deserialize cell values as `chrono::NaiveDateTime`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_datetime`] method to the cell value, and returns
/// `Ok(Some(value_as_datetime))` if successful or `Ok(None)` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_datetime_or_none<'de, D>(
deserializer: D,
) -> Result<Option<chrono::NaiveDateTime>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_datetime())
}

/// A helper function to deserialize cell values as `chrono::NaiveDateTime`,
/// useful when cells may also contain invalid values (i.e. strings).
/// It applies the [`as_datetime`] method to the cell value, and returns
/// `Ok(Ok(value_as_datetime))` if successful or `Ok(Err(value_to_string))` if unsuccessful,
/// therefore never failing. This function is intended to be used with Serde's
/// [`deserialize_with`](https://serde.rs/field-attrs.html) field attribute.
#[cfg(feature = "dates")]
pub fn deserialize_as_datetime_or_string<'de, D>(
deserializer: D,
) -> Result<Result<chrono::NaiveDateTime, String>, D::Error>
where
D: Deserializer<'de>,
{
let data = Data::deserialize(deserializer)?;
Ok(data.as_datetime().ok_or_else(|| data.to_string()))
}