Records in DSS file written to SHG/Albers is corrupt

[openSourcerer9000]

So I'm trying to write a DSS file in SHG. The data is in EPSG:5070 with 20m cell size, snapped to the SHG grid (all x and y values are multiples of 20). This is what I'm putting into my grid info:

t =  ds.rio.transform()
# rioxarray method, returns
# Affine(20.0, 0.0, 251370.0,
#           0.0, -20.0, 903530.0)

grid_info.update([('grid_type','albers'),
                ('grid_crs',ds.spatial_ref.crs_wkt),
                ('grid_transform',t,
                ('data_type','inst-val'),
                ('data_units','in'),
                ('opt_time_stamped',False),
                ('opt_lower_left_x',int( t[2]/np.abs(t[0]) ) ),
                ('opt_lower_left_y',int( t[5]/np.abs(t[4]) ) )
                ])

When I use 'grid_type':'specified', it produces a valid DSS, but HMS complains that my gridded deficit and constant loss params are bad, which I am assuming is due to the projection. When I try to specify 'albers' or 'shg', I can view the record names in DSSVue, but when I try to plot or tabulate nothing comes up. In the HMS DSS viewer it pops up an error 'there is no record' when I try to select the visible pathnames.

I'm thinking it may be something to do with this operation in grid_accessors.py, I'm not sure:

            if gridinfo['grid_type'].lower() in ('albers','albers-time','shg','shg-time'):
                gridinfo = correct_shg_gridinfo(gridinfo,out_data[0].shape)

I was able to add the same grids through vortex (and run the HMS method successfully), so I read those into pydsstools and their grid_type was actually 'albers-time'. Trying 'albers-time' in my own results in this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
~\Anaconda3\envs\raz\lib\site-packages\pydsstools\heclib\dss\HecDss.py in put_grid(self, pathname, data, profile, flipud, compute_range, inplace, raise_profile_error)
    435                     # TODO: Found out HecTime('1') passes this test
--> 436                     HecTime(dpart)
    437                     HecTime(epart)

~\Anaconda3\envs\raz\lib\site-packages\pydsstools\src\hectime.pyx in pydsstools._lib.x64.core_heclib.HecTime.__init__()

~\Anaconda3\envs\raz\lib\site-packages\pydsstools\src\hectime.pyx in pydsstools._lib.x64.core_heclib.HecTime.parse_datetime_string()

AttributeError: 'NoneType' object has no attribute 'year'

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_19644/125238972.py in <module>
      1 get_ipython().run_line_magic('run', '-n C:\\Users\\seanm\\Docs\\LWI\\DefCon\\gssurgo_to_defcon_grids')
----> 2 dss.rioxdsToDSS(ds,outDSS,parts)

~\Docs\LWI\DefCon\gssurgo_to_defcon_grids.py in rioxdsToDSS(ds, outDSSpth, parts)
     90                 print(pathname)
     91                 if 'band' in ds.coords and 'band' in ds.dims:
---> 92                     fid.put_grid(pathname,ds.sel(band=ds['band'].values[0]).drop('band')[varr].values,grid_info)
     93                 else:
     94                     fid.put_grid(pathname,ds[varr].values,grid_info)

~\Anaconda3\envs\raz\lib\site-packages\pydsstools\heclib\dss\HecDss.py in put_grid(self, pathname, data, profile, flipud, compute_range, inplace, raise_profile_error)
    437                     HecTime(epart)
    438                 except:
--> 439                     raise Exception('For %s grid type, DPart and EPart of pathname must be datetime string')
    440                 else:
    441                     grid_info['opt_time_stamped'] = 1

Exception: For %s grid type, DPart and EPart of pathname must be datetime string

py 3.9
pydsstools 2.2

[gyanz]

@openSourcerer9000 Based on #31, HEC-HMS does not support HEC-DSS 7 format.

[openSourcerer9000]

Ouch

WARNING:root:Writing DSS grid record in DSS-6 file is not supported

So pydsstools and HMS are just not compatible? Is there a legacy pydsstools version which will write these to a DSS-6? Or is there some way to convert a grid in DSS 7 to DSS 6?

I think beyond that, there is still something going on with this Albers projection, as DSSVue isn't able to read the grids either, yet their pathnames still show up in the list.

[danhamill]

I have not had any success converting grids in a version 7 dss file to version 6. I would suggest using the HEC utility vortex for your HMS grid processing needs. I believe the vortex tools come with HMS versions >= 4.9.

[steezbert]

@openSourcerer9000 from the errors I'd say you should check if the filepath parts are all correct, especially if using 'time' grids. What I don't get is why would you use the 'time' grid version for parameters?

One error message is AttributeError: 'NoneType' object has no attribute 'year', so I would check that.

The error Exception: For %s grid type, DPart and EPart of pathname must be datetime string also comes from wrong date formats in the internal dss filepath.
I had issues with the later error as well, but with the latest version installed from wheel they seem to be fixed (py 3.8, pydsstools 2.2). However I still get the errors from #31 when trying to use the dss files with HEC-HMS.

As mentioned above, HEC vortex is a solid solution to get your data from different raster file formats into dss files. The only thing they sadly don't support right now is Geotiff raster stacks. From them I also know, that HEC-DSS 7 format should work with HEC-HMS, so the error has to come from something else.

Also have you checked if all georeferenced data is using the the reference system and projection?

I still hope to find a solution to write valid grids with with pydsstools, as I like it's functionality especially when it comes to large data sets.

[gyanz]

I have contacted HEC to resolve this issue. I will provide an update about my findings. Thanks everybody for your feedback.

[gyanz]

Based on #36, HEC-HMS supports DSS 7. I will review the grid writer in Vortex and try to emulate that in pydsstools.

[gyanz]

Can anybody provide me grid data (created using Vortex or another software) that works in HEC-HMS?