Allow CF compliant NetCDF (not just NSIDC compliant)

[lisakaser]

Sample data: IRTIT3
 (does not have time component in file - file names have date in them)

Options:

• use collection level temporal range
• use file name that has a date
• ask for Ops input in ini file

Acceptance criteria:
If we ingest CF compliant NetCDF (not NSIDC) we can still generate UMM-G

[juliacollins]

Test data are available at /disks/sidads_staging/SAMPLE_DATA/DUCk_project/IRTIT3_DUCk

[juliacollins]

@afitzgerrell Here is some example output from my ugly code! I've also attached the file example.ini.txt so you could review my approach to declaring the attributes not available in the netCDF data files. (Had to give the file a .txt suffix so Github would let me upload it.)

IRTIT3_20101120_PineIsland.nc.json
IRTIT3_20101120_PineIsland.nc.cnm.json
example.ini.txt

[juliacollins]

Pull request exists (#114) but there is still some cleanup to be done and documentation to add.

[juliacollins]

Remaining UMM-G outputs for @afitzgerrell review

IRTIT3_20110407_Umanaq.nc.json
IRTIT3_20110413_Russell.nc.json
IRTIT3_20120421_Jakobshavn.nc.json
IRTIT3_20130420_Humboldt.nc.json

I added a comment to issue-111 but probably should have kept all the discussion here. See: #111 (comment)

[afitzgerrell]

Hi Julia,

RE: bullet 2 (file name contains a date)--
Aside from pixel size and version*, all of these look good in terms of reflecting the .ini appropriately in the UMM-G files. The UMM-G files are all formatted as expected and contain the elements expected.

*in a perfect world, more files in the future will include the collection version in their names too to help declare version successfully. This particular data set is at version 2 (ditto for a specific time or temporal range! however as discussed in #111 your use of the file name + regex to turn the date into the file start time and declare its duration 23:59:59 is fine and spiffy).

RE: bullet 3 (OPS input required to ini file)--
IN case we were tp encounter some file name craziness where there isn't a date in the name (this should almost never happen, so may be moot), would the .ini file be able to contain two elements conveying BeginningDateTime and EndingDateTime verbaim?

[juliacollins]

pig.geojson.txt

@afitzgerrell Attached is my latest geojson representing the PIG :-) location. It appears to be roughly in the right place, but I still need to compare it to the raw location data in detail to see fixing the offset was all that was needed.

[juliacollins]

@afitzgerrell Here's an updated PIG geojson. Note! It does not have any padding added (the 12.5 meters). I'll re-instate that bit of code and generate a new geojson asap.

pig.geojson.txt

[juliacollins]

@afitzgerrell ! Here is a PIG with padding! pig_with_padding.geojson.txt

[juliacollins]

@afitzgerrell -- attaching a geojson for one of the 0081 files so you can verify it still looks correct. Also, let me know if you want something besides geojson. I'm finally at a place where I can create some actually UMM-G again!
interp0081.geojson.txt

[juliacollins]

@afitzgerrell at last some actual UMM-G and CNM output to review.

IRTIT3_20101120_PineIsland.nc.cnm.json
IRTIT3_20110407_Umanaq.nc.cnm.json
IRTIT3_20110413_Russell.nc.cnm.json
IRTIT3_20120421_Jakobshavn.nc.cnm.json
IRTIT3_20130420_Humboldt.nc.cnm.json

IRTIT3_20101120_PineIsland.nc.json
IRTIT3_20110407_Umanaq.nc.json
IRTIT3_20110413_Russell.nc.json
IRTIT3_20120421_Jakobshavn.nc.json
IRTIT3_20130420_Humboldt.nc.json

And for comparison/confirmation, a few NSIDC-0081 files:

NSIDC0081_SEAICE_PS_N25km_20211101_v2.0_DUCk.nc.cnm.json
NSIDC0081_SEAICE_PS_N25km_20211101_v2.0_DUCk.nc.json

NSIDC0081_SEAICE_PS_S25km_20211102_v2.0_DUCk.nc.cnm.json
NSIDC0081_SEAICE_PS_S25km_20211102_v2.0_DUCk.nc.json

[afitzgerrell]

first things first, i checked pig_with_padding and it looks perfect. i can honestly say "no notes" 😃.

i'll get on w/checking everything else now!

[afitzgerrell]

latest update: spatial representations from within the latest 0081 ummg files attached above are spot-on for both north and south polar stereo files.

note: the coordinate values in the ummg PS North files differ slightly from those in the interp0081.geojson.txt julia added above yesterday morning. the coordinate locations in that text file show an approx 9775 m x / y shift from the actual data corners. i'm just relaying this for information, not because it's necessarily a problem since the ummg files from julia yesterday afternoon are those wherein the coordinates are spot-on, so i'm assuming it's expected evolution.

i'm moving onto irtit3 qc now.

[afitzgerrell]

Alrighty. I've looked at the spatial representation coordinates and they're great for all of the IRTIT3 glacier files' data extent. the content of the cnm and ummg are up to snuff as well. Thus it appears that metgenc's ability—between it using a file name that has a date and taking information input in ini file—will indeed spit out ingest-worthy file-level metadata and cnm messages. I'll hop back over to 111 now and we can await end-2-end testing of this when the time is right.

[juliacollins]

@afitzgerrell I took a stab at some very minimal documentation of our optional ini elements. You can see the README in the pull request at https://github.com/nsidc/granule-metgen/tree/issue-104?tab=readme-ov-file if you want to take a look.

[afitzgerrell]

I've taken a look and I think this is a fine start, and as Kevin mentioned in Slack, we can add to it as necessary / move it or sections around as necessary (and thus it's good for now)!