Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(createpackages): use jinja for mf6 module code generation #2333

Open
wants to merge 95 commits into
base: develop
Choose a base branch
from

Conversation

wpbonelli
Copy link
Member

@wpbonelli wpbonelli commented Oct 10, 2024

Carved out of #2317. Just move to Jinja as a first step. Type hints can come afterwards as they will take some effort to get right. The aim here is to match the old createpackages.py capabilities without relying on mfstructure.py. This is an initial step to trim mfstructure.py in favor of a leaner representation of the input spec. mfstructure.py is still used at runtime which will need to be unraveled in followup work.

Introduce a flopy.mf6.utils.codegen module with some new machinery for generating code with Jinja. This includes a bunch of filters handling quirks of the generated classes which we can aim to gradually eliminate. The templates should also get simpler over time.

The codegen module consumes TOML rather than legacy DFNs. Devtools is used at code generation time to first convert DFNs to TOML. This happens under the hood, with no change in apparent behavior to the user of the codegen utility.

Jinja2 and modflow-devtools are added to a new optional dependency group codegen. These are required to use the code generation utilities. These as well as transitive dependencies are added to environment.yml.

Miscellaneous:

@wpbonelli wpbonelli force-pushed the jinja branch 2 times, most recently from a3b7c2e to af1e7a5 Compare October 10, 2024 17:49
@wpbonelli wpbonelli added modflow 6 dependencies Pull requests that update a dependency file maintenance labels Oct 10, 2024
@wpbonelli wpbonelli force-pushed the jinja branch 4 times, most recently from d1e8a21 to a7f2644 Compare October 10, 2024 18:30
@wpbonelli wpbonelli added this to the 3.9 milestone Oct 10, 2024
Copy link
Contributor

@deltamarnix deltamarnix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked through everything, but I left you a PR on your own branch with indications on how I would probably approach this. Let's take a look at it together and see if we can get rid of the shim :)

@wpbonelli wpbonelli force-pushed the jinja branch 2 times, most recently from bab83ac to cf68432 Compare October 18, 2024 00:32
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

Attention: Patch coverage is 4.91803% with 406 lines in your changes missing coverage. Please review.

Project coverage is 76.0%. Comparing base (bb9824e) to head (5c2a043).
Report is 48 commits behind head on develop.

Files with missing lines Patch % Lines
flopy/mf6/utils/codegen/filters.py 0.0% 308 Missing ⚠️
flopy/mf6/utils/codegen/__init__.py 0.0% 70 Missing ⚠️
flopy/mf6/utils/codegen/component.py 0.0% 20 Missing ⚠️
flopy/mf6/utils/generate_classes.py 40.0% 6 Missing ⚠️
flopy/mf6/utils/createpackages.py 50.0% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           develop   #2333     +/-   ##
=========================================
+ Coverage     68.4%   76.0%   +7.6%     
=========================================
  Files          294     296      +2     
  Lines        59390   63717   +4327     
=========================================
+ Hits         40652   48474   +7822     
+ Misses       18738   15243   -3495     
Files with missing lines Coverage Δ
flopy/mf6/data/mfdatastorage.py 75.4% <ø> (+6.5%) ⬆️
flopy/mf6/data/mfdatautil.py 77.9% <ø> (+23.3%) ⬆️
flopy/mf6/data/mfstructure.py 73.6% <100.0%> (+9.5%) ⬆️
flopy/mf6/mfbase.py 87.6% <100.0%> (+12.1%) ⬆️
flopy/mf6/mfpackage.py 81.9% <ø> (+17.3%) ⬆️
flopy/mf6/mfsimbase.py 75.6% <100.0%> (+13.4%) ⬆️
flopy/mf6/utils/createpackages.py 60.0% <50.0%> (+53.5%) ⬆️
flopy/mf6/utils/generate_classes.py 19.2% <40.0%> (+1.4%) ⬆️
flopy/mf6/utils/codegen/component.py 0.0% <0.0%> (ø)
flopy/mf6/utils/codegen/__init__.py 0.0% <0.0%> (ø)
... and 1 more

... and 246 files with indirect coverage changes

Copy link
Contributor

@deltamarnix deltamarnix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've only looked through some of the output files, but I have found some remarkable differences that we should check.

@wpbonelli
Copy link
Member Author

Thanks. Yeah, this is why I want to wait to merge til after release. I think it'll take some time to ferret out all the minute differences and make sure no behavior has changed, even after merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file maintenance modflow 6
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants