[1pt] Improve handling of test sites and duplicate sites in CatFIM, update CatFIM vis #1450
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, the restricted sites list had a bunch of test sites but they were only being restricted for stage-based CatFIM. I updated the list so all test sites are excluded from BOTH stage-based and flow-based CatFIM. I also updated CatFIM so that when a site is excluded due to being on the restricted sites list, the phrase "Restricted Site" is included in the status. This will make it easier to understand why sites are removed.
I've also updated the CatFIM mapping functions so that there are a few functions that save the output plot into a .png file. This streamlines our ability to analyze CatFIM outputs and compare the effects of code changes.
Finally, I've updated the APHS metadata retrieval code so that when it checks for and removes duplicate site IDs, it also prints a list of the duplicate site IDs that were removed (as well as a count for how many sites were removed because they lack a site ID).
Changes
tools/catfim/ahps_restricted_sites.csv
: Updated the restricted sites list so the test sites are applied to both stage- and flow-based CatFIM. Tidied up status phrasing.tools/catfim/generate_categorical_fim.py
: Updated restricted site processing so "Restricted Site" is appended at the beginning of the site status for sites that are removed due to the restricted sites list.tools/catfim/generate_categorical_fim_flows.py
: Updated restricted site processing so "Restricted Site" is appended at the beginning of the site status for sites that are removed due to the restricted sites list. Also updated the metadata retrieval code so it now prints the ID's of sites excluded due to being duplicates.tools/catfim/vis_categorical_fim.py
: Update the CatFIM mapping functions to include two functions for saving CatFIM plots. Cleaned up comments and corrected code usage examples.Testing
I've tested this on flow- and stage-based CatFIM for the following HUCs/Sites:
Additional areas for testing would be other HUCs that have restricted sites:
These sites previously had mapped CatFIM for flow-based but with these updates they should not.
Deployment Plan (For developer use)
How does the changes affect the product?
Issuer Checklist (For developer use)
You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.
[_pt] PR: <description>
dev
branch (the default branch), you have a descriptive Feature Branch name using the format:dev-<description-of-change>
(e.g.dev-revise-levee-masking
)dev
branchpre-commit
hooks were run locally4.x.x.x
Merge Checklist (For Technical Lead use only)