-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exposure filtering script #85
Conversation
…ion and updating exposure call in notebook
Spatial and Temporal functions isolated from main exposure function call. Reduced modelling to ds scale and removing pixel based calculations from filtered workflows. I think we should keep all-time exposure calculation as pixel based to match the continental workflows. TODO: change day/night filtering to UTC time and remove regional sampling of datasets.
TODO: test outputs to ensure sensible regional day/night assignments of UTC datetimes, incorporate 'exposure_edited.py' into master 'exposure.py' script, PR changes into main
todo: propagate changes to function call through notebooks and elevation.py
todo: update changes into exposure.py and change func load calls in elevation.py, CLI notebook and workflow notebook
… as intertidal_workflow.ipynb PR changes document significantly revised exposure.py workflow and simplification of the exposure computation for custom/filtered workflows
remove script and notebook used during exposure filter testing
For full integration test results, refer to the Tests directory README. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #85 +/- ##
=======================================
- Coverage 65.5% 57.0% -8.6%
=======================================
Files 8 8
Lines 714 887 +173
=======================================
+ Hits 468 506 +38
- Misses 246 381 +135 ☔ View full report in Codecov by Sentry. |
Todo: fix filtered exposure outputs which are no longer working after updating tidal model calls
…ing. Also tested a fix for calculating high/lowtide exposure in diurnal regimes. TODO: apply the fix to temporal modelling. TODO: implement inequality testing on the peak detection to identify diurnal regimes and simplify exposure calculation to use low_peaks2 (or high_peaks2) in these settings accordingly
…e only which streamlines the function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @erialC-P, thanks so much for all the changes you've made - this version is super super nice. I've pushed a few additional changes myself, mainly:
- Black formatting and added docstrings
- Updated all filter options to
lowercase_snakes
- Refactored the tide modelling code so tide modelling only ever happens once
- Updated
spatial_filters
andtemporal_filters
to no longer pass externally created objects in and out of the function - now only the outputs are passed out and then assigned into the external objects outside of the functions - Some small changes to variable names and docstrings (e.g.
xarray.Datasets
were being referred to as dicts in some parts of the code, which I think was probably a legacy of older versions
I think it's pretty much ready to go now as-is, but here's a few other things that I think would be worth focusing a bit of time on if you have capacity:
- As discussed on Teams, it would be really nice if both
temporal_filters
andspatial_filters
returned exactly the same output: just a filtered list of timesteps. These outputs could then be handled nicely in one single place outside of those functions using the same consistent code, avoiding duplication and ensuring that exactly the same process/logic is applied to every filter, regardless of its type. I think this could lead to cleaner code and a more consistent user experience overall. - If you wanted to, you could probably think about pulling out some parts of
spatial_filters
into even smaller sub-functions (e.g. things like the neap/low/high bits where similar code is repeated a few times), and call those subfuncs inside relevant parts ofspatial_filters
(would make it super easy to update those methods in the future as you could just tweak the mini sub-function - I think there's quite a few places where docstrings haven't been updated for updated code. Would be worth going through the script carefully before we merge just to make sure all the doco is up-to-date.
- I don't personally like the name "spatial" in
spatial_filters
. I think it's potentially going to add confusion as it has such geographic connotations... would just something like "tidal_filters" be beter? Or "tidalstage_filters"? - Also as discussed, it could be neat if we could do away with
combined_filters
and instead just usefilters
, perhaps letting users pass in tuples of filters inside the list. But this is very much a "nice to have" and not a major priority. 🙂
Removed a outstanding references to spatial filtering and ensured script is up to date and consistent with closed PR #85
This PR includes significant changes to the exposure calculation and custom filtering workflow, based on really helpful review and feedback from the first version.
Major changes to the workflow include the following:
Incorporation of recommendations from the initial review of this workflow have helped to: