Merge pull request #13 from NWC-CUAHSI-Summer-Institute/add-NN

Add nn

README.md

@@ -2,6 +2,8 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) ![versions](https://img.shields.io/pypi/pyversions/hydra-core.svg) [![CodeStyle](https://img.shields.io/badge/code%20style-Black-black)]()
 
 The differentiable parameter learning Conceptual Functional Equivalent (dCFE) is a differentiable implementation of CFE (see below). All operations of the model are coded using PyTorch to track gradients and tune model parameters. Currently the model allows gradient tracking from runoff to 9 calibration parameters (```bb```, ```satdk```, ```smcmax```, ```slop```, ```Cgw```, ```expon```, ```max_gw_storage```, ```K_nash```, ```K_lf```).
+
+Currently the model takes only static attributes from one basin. Development in progress to include dynamic parameters for multiple CAMELS basins. 
 
 #### Conceptual Functional Equivalent (CFE) Model
 The CFE model is designed to be a simplified and functionaly equivalent model of the National Water Model. The model code was originally written by Dr. Fred Ogden and converted to BMI-compliant format in the Next-Gen framework by NOAA-OWP. The official CFE code by Dr. Fred Oden and NOAA-OWP lives [here](https://github.com/NOAA-OWP/cfe/). [The Python version of the code](https://github.com/NWC-CUAHSI-Summer-Institute/cfe_py) is developed for the prototyping, research, and development. This code is developed upon the Python version and for research purpose. 
@@ -19,21 +21,25 @@ We are using [Hydra](https://github.com/facebookresearch/hydra) to store/manage
 
 The main branch code is currently configured to run the CAMELS basin id #0102500 test case. If you want to use your own case, you will need to manage three config files located here:
 
-- ```dCFE/config.yaml```
- - The main config file. I recommend looking at the Hydra config docs here to learn how this file is structured.
+- ```dCFE/src/config.yaml```
+ - The main config file. Choose appropriate ```run_type```, ```soil_scheme```, and ```basin_id``` here. 
+ - ```run_type```
+ - ```generate_synthetic```: to generate synthetic data. Make sure to change the ```synthetic``` configs. 
+ - ```ML```: to run and train the dCFE against observed data
+ - ```ML_synthetic_test```: to run and test the dCFE against the synthetic data generated after ```generate_synthetic```
 - ```dCFE/src/models/config/base.yaml```
  - This holds all config values for the models
 - ```dCFE/src/data/config/<site_name>.yaml```
  - This holds all config values for the dataset you're working on.
 - ```dCFE/data/```
  - This holds all forcing (P, PET) and validation dataset (runoff)
 
-To run the code, just run the following command inside the dCFE/ folder:
+To run the code, just run the following command inside the dCFE/src folder:
 
-```python __main__.py```
+```python .```
 
 ## Gradient chain visualization
-See [this PDF](https://drive.google.com/open?id=1vO2RKhXwjaQ_-D24OGIcZiPow6RE33v1&usp=drive_fs) for an example of CFE gradient chain. 
+See [this PDF](https://github.com/NWC-CUAHSI-Summer-Institute/ngen-aridity/blob/main/CFE_gradient_chain_demo.pdf) for an example of CFE gradient chain. 
 
 ## Contributing
 

data/basin_ids.txt

@@ -0,0 +1,3 @@
+01022500
+01031500
+01137500

data/cat_01022500_bmi_config_cfe.json

@@ -0,0 +1,39 @@
+{
+ "soil_params": {
+ "depth": 2.0,
+ "bb": 8.013513513513514,
+ "satdk": 1.3156783783783787e-05,
+ "satpsi": 0.1647076737162162,
+ "slop": 0.08824091635135137,
+ "smcmax": 0.37300223004054056,
+ "wltsmc": 0.04966811960810811,
+ "D": 2.0,
+ "mult": 1000.0
+ },
+ "forcing_file": "../data/camels/gauch_etal_2020/nldas_hourly/01022500_hourly_nldas.csv",
+ "catchment_area_km2": 573.6,
+ "refkdt": 3.8266861353378374,
+ "max_gw_storage": 0.021342666010108112,
+ "Cgw": 1.8e-05,
+ "expon": 6.72972972972973,
+ "gw_storage": 0.05,
+ "alpha_fc": 0.33,
+ "K_nash": 0.03,
+ "K_lf": 0.01,
+ "nash_storage": [
+ 0.0,
+ 0.0
+ ],
+ "giuh_ordinates": [
+ 0.93,
+ 0.06,
+ 0.01,
+ 0.0,
+ 0.0
+ ],
+ "stand_alone": 1,
+ "unit_test": 0,
+ "compare_results_file": "",
+ "partition_scheme": "Schaake",
+ "soil_scheme": "classic"
+}

data/cat_01031500_bmi_config_cfe.json

@@ -0,0 +1,43 @@
+{
+ "soil_params": {
+ "depth": 2.0,
+ "bb": 8.637254901960784,
+ "satdk": 1.6051127450980392e-05,
+ "satpsi": 0.5111765637352941,
+ "slop": 0.03758763542156863,
+ "smcmax": 0.4564483472745098,
+ "wltsmc": 0.07387927075490196,
+ "D": 2.0,
+ "mult": 1000.0
+ },
+ "forcing_file": "../data/camels/gauch_etal_2020/nldas_hourly/01031500_hourly_nldas.csv",
+ "catchment_area_km2": 769.05,
+ "refkdt": 2.753617570666666,
+ "max_gw_storage": 0.052947464611588245,
+ "Cgw": 1.8e-05,
+ "expon": 4.382352941176471,
+ "gw_storage": 0.05,
+ "alpha_fc": 0.33,
+ "K_nash": 0.03,
+ "K_lf": 0.01,
+ "nash_storage": [
+ 0.0,
+ 0.0
+ ],
+ "giuh_ordinates": [
+ 0.39,
+ 0.29,
+ 0.18,
+ 0.09,
+ 0.03,
+ 0.01,
+ 0.01,
+ 0.0,
+ 0.0
+ ],
+ "stand_alone": 1,
+ "unit_test": 0,
+ "compare_results_file": "",
+ "partition_scheme": "Schaake",
+ "soil_scheme": "classic"
+}

data/cat_01137500_bmi_config_cfe.json

@@ -0,0 +1,49 @@
+{
+ "soil_params": {
+ "depth": 2.0,
+ "bb": 2.0,
+ "satdk": 2.9887654320987658e-06,
+ "satpsi": 0.12760831659259259,
+ "slop": 0.020431365913580246,
+ "smcmax": 0.483040294382716,
+ "wltsmc": 0.04628188386419752,
+ "D": 2.0,
+ "mult": 1000.0
+ },
+ "forcing_file": "../data/camels/gauch_etal_2020/nldas_hourly/01047000_hourly_nldas.csv",
+ "catchment_area_km2": 909.1,
+ "refkdt": 0.4832764430740741,
+ "max_gw_storage": 0.15402919006299998,
+ "Cgw": 1.8e-05,
+ "expon": 3.0,
+ "gw_storage": 0.05,
+ "alpha_fc": 0.33,
+ "K_nash": 0.03,
+ "K_lf": 0.01,
+ "nash_storage": [
+ 0.0,
+ 0.0
+ ],
+ "giuh_ordinates": [
+ 0.25,
+ 0.23,
+ 0.18,
+ 0.13,
+ 0.09,
+ 0.05,
+ 0.03,
+ 0.02,
+ 0.01,
+ 0.01,
+ 0.0,
+ 0.0,
+ 0.0,
+ 0.0,
+ 0.0
+ ],
+ "stand_alone": 1,
+ "unit_test": 0,
+ "compare_results_file": "",
+ "partition_scheme": "Schaake",
+ "soil_scheme": "classic"
+}