|
1 | 1 | # xtclim
|
| 2 | + |
2 | 3 | ## ML-based extreme events detection and characterization (CERFACS)
|
3 | 4 |
|
4 | 5 | The code is adapted from CERFACS' [repository](https://github.com/cerfacs-globc/xtclim/tree/master).
|
5 |
| -The implementation of a pipeline with itwinai framework is shown below. |
| 6 | +The implementation of a pipeline with itwinai framework is shown below. |
| 7 | + |
| 8 | +## Method |
6 | 9 |
|
7 |
| -## Method |
8 | 10 | Convolutional Variational AutoEncoder.
|
9 | 11 |
|
10 | 12 | ## Input
|
| 13 | + |
11 | 14 | "3D daily images", daily screenshots of Europe for three climate variables (maximum temperature, precipitation, wind).
|
12 | 15 |
|
13 |
| -## Output |
| 16 | +## Output |
| 17 | + |
14 | 18 | Error between original and reconstructed image: postprocessed for analysis in the `scenario_season_comparison.ipynb` file.
|
15 | 19 |
|
16 |
| -## Idea |
| 20 | +## Idea |
| 21 | + |
17 | 22 | The more unusual an image (anomaly), the higher error.
|
18 | 23 |
|
19 | 24 | ## Information on files
|
20 | 25 |
|
21 |
| -In the preprocessing folder, the `preprocess_functions_2d_ssp.py` class loads NetCDF files from a `data` folder, which has to be specified in `dataset_root` in the config file `pipeline.yaml` (please change the location). The data can be found [here](https://b2drop.eudat.eu/s/rtAadDNYDWBkxjJ). The given class normalizes and adjusts the data for the network. The function `preprocess_2d_seasons.py` splits the data into seasonal files. Preprocessed data is stored in the `input` folder. |
| 26 | +In the preprocessing folder, the `preprocess_functions_2d_ssp.py` class loads NetCDF files from a `data` folder, |
| 27 | +which has to be specified in `dataset_root` in the config file `config.yaml` (please change the location). |
| 28 | +The data can be found [here](https://b2drop.eudat.eu/s/rtAadDNYDWBkxjJ). The given class normalizes |
| 29 | +and adjusts the data for the network. The function `preprocess_2d_seasons.py` splits the data into |
| 30 | +seasonal files. Preprocessed data is stored in the `input` folder. |
22 | 31 |
|
23 |
| -The file `train.py` trains the network. Caution: It will overwrite the weights of the network already saved in outputs (unless you change the path name `outputs/cvae_model_3d.pth` in the script). |
| 32 | +The file `train.py` trains the network. Caution: It will overwrite the weights of the network already |
| 33 | +saved in outputs (unless you change the path name `outputs/cvae_model_3d.pth` in the script). |
24 | 34 |
|
25 | 35 | The `anomaly.py` file evaluates the network on the available datasets - train, test, and projection.
|
26 | 36 |
|
27 |
| -## How to launch pipeline |
| 37 | +## How to launch training workflow |
28 | 38 |
|
29 |
| -The config file `pipeline.yaml` contains all the steps to execute the workflow. You can launch it from the root of the repository with: |
| 39 | +The config file `config.yaml` contains all the steps to execute the workflow. |
| 40 | +You can launch it from the root of the repository with: |
30 | 41 |
|
31 | 42 | ```bash
|
32 |
| -python train.py -p pipeline.yaml |
33 |
| - |
| 43 | +itwinai exec-pipeline --config config.yaml |
34 | 44 | ```
|
35 | 45 |
|
36 | 46 | ## TODOs
|
| 47 | + |
37 | 48 | Integration of post-processing step + distributed strategies
|
0 commit comments