Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1pt] Improve handling of test sites and duplicate sites in CatFIM, update CatFIM vis #1450

Open
wants to merge 9 commits into
base: dev
Choose a base branch
from

Conversation

EmilyDeardorff
Copy link
Contributor

@EmilyDeardorff EmilyDeardorff commented Mar 5, 2025

Previously, the restricted sites list had a bunch of test sites but they were only being restricted for stage-based CatFIM. I updated the list so all test sites are excluded from BOTH stage-based and flow-based CatFIM. I also updated CatFIM so that when a site is excluded due to being on the restricted sites list, the phrase "Restricted Site" is included in the status. This will make it easier to understand why sites are removed.

I've also updated the CatFIM mapping functions so that there are a few functions that save the output plot into a .png file. This streamlines our ability to analyze CatFIM outputs and compare the effects of code changes.

Finally, I've updated the APHS metadata retrieval code so that when it checks for and removes duplicate site IDs, it also prints a list of the duplicate site IDs that were removed (as well as a count for how many sites were removed because they lack a site ID).

Changes

  • tools/catfim/ahps_restricted_sites.csv: Updated the restricted sites list so the test sites are applied to both stage- and flow-based CatFIM. Tidied up status phrasing.

  • tools/catfim/generate_categorical_fim.py: Updated restricted site processing so "Restricted Site" is appended at the beginning of the site status for sites that are removed due to the restricted sites list.

  • tools/catfim/generate_categorical_fim_flows.py: Updated restricted site processing so "Restricted Site" is appended at the beginning of the site status for sites that are removed due to the restricted sites list. Also updated the metadata retrieval code so it now prints the ID's of sites excluded due to being duplicates.

  • tools/catfim/vis_categorical_fim.py: Update the CatFIM mapping functions to include two functions for saving CatFIM plots. Cleaned up comments and corrected code usage examples.

Testing

I've tested this on flow- and stage-based CatFIM for the following HUCs/Sites:

HUC8 Site(s) of Interest
17110018 test3, test4
16030006 test9
04140201 ondn6

Additional areas for testing would be other HUCs that have restricted sites:

HUC8 Site(s) of Interest
16020101 test
17110018 test4
04050004 jtest

These sites previously had mapped CatFIM for flow-based but with these updates they should not.

Deployment Plan (For developer use)

How does the changes affect the product?

  • Code only?
  • If applicable, has a deployment plan be created with the deployment person/team?
  • Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
  • If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
  • Require new pre-clip set?
  • Has new or updated python packages?

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

  • Informative and human-readable title, using the format: [_pt] PR: <description>
  • Links are provided if this PR resolves an issue, or depends on another other PR
  • If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
  • Changes are limited to a single goal (no scope creep)
  • The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
  • pre-commit hooks were run locally
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • CHANGELOG updated with template version number, e.g. 4.x.x.x
  • Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

  • Update CHANGELOG with latest version number and merge date
  • Update the Citation.cff file to reflect the latest version number in the CHANGELOG
  • If applicable, update README with major alterations

@EmilyDeardorff EmilyDeardorff self-assigned this Mar 5, 2025
@EmilyDeardorff EmilyDeardorff added the CatFIM NWS Flood Categorical HAND FIM label Mar 6, 2025
@EmilyDeardorff EmilyDeardorff changed the title WIP [1pt] Improve handling of test sites and duplicate sites in CatFIM WIP [1pt] Improve handling of test sites and duplicate sites in CatFIM, update CatFIM vis Mar 10, 2025
@EmilyDeardorff EmilyDeardorff linked an issue Mar 10, 2025 that may be closed by this pull request
@EmilyDeardorff EmilyDeardorff changed the title WIP [1pt] Improve handling of test sites and duplicate sites in CatFIM, update CatFIM vis [1pt] Improve handling of test sites and duplicate sites in CatFIM, update CatFIM vis Mar 11, 2025
@EmilyDeardorff EmilyDeardorff marked this pull request as ready for review March 11, 2025 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CatFIM NWS Flood Categorical HAND FIM
Projects
None yet
2 participants