Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fillval does not allow string inputs for string variables #995

Open
jklenzing opened this issue Apr 25, 2022 · 2 comments
Open

BUG: Fillval does not allow string inputs for string variables #995

jklenzing opened this issue Apr 25, 2022 · 2 comments
Milestone

Comments

@jklenzing
Copy link
Member

Describe the bug
Some datasets include strings in the data. An example is the channel, hemisphere, and scan information in the GOLD dataset. A numeric FillVal does not make sense here, but pysat requires the FillVal to be numeric.

To Reproduce

Load a GOLD NMAX file with xarray, inspect the variable "channel":

<xarray.DataArray 'channel' (nscans: 24)>
array([b'CHB', b'CHB', b'CHB', b'CHB', b'CHB', b'CHB', b'CHB', b'CHB',
       b'CHB', b'CHB', b'CHB', b'CHB', b'CHB', b'CHA', b'CHB', b'CHA',
       b'CHA', b'CHB', b'CHA', b'CHB', b'CHA', b'CHB', b'CHB', b'CHA'],
      dtype='|S50')
Dimensions without coordinates: nscans
Attributes:
    FORMAT:    A1
    FILLVAL:   *
    FIELDNAM:  Channel
    DEPEND_0:  nscans
    CATDESC:   Channel identifier ("A" or "B")
    VAR_TYPE:  metadata

Load the same file with pysat. The FILLVAL is dropped.

Expected behavior
pysat should check that the fillval type matches the format.

Additional context
discovered while working on pysat/pysatNASA#110

Seeing both in develop and in #983.

@rstoneback
Copy link
Collaborator

pysat Meta currently only supports a single data type per label. Expanding support would require updating the underlying data format. I think we probably do need better type support but not the fastest of fixes.

@jklenzing
Copy link
Member Author

Agreed. Just wanted to document this for the next version.

@rstoneback rstoneback added this to the 3.2.0 Release milestone May 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants