Skip to content

Commit 89f45b9

Browse files
author
Klaus Zimmermann
committed
Reduce formatting changes
1 parent c8040ef commit 89f45b9

File tree

3 files changed

+56
-60
lines changed

3 files changed

+56
-60
lines changed

doc/develop/fixing_data.rst

Lines changed: 45 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -10,32 +10,31 @@ data that is read using
1010
ESMValCore also allows for some departures with compliance (see
1111
:ref:`cmor_check_strictness`). Beyond that situation, some datasets
1212
(either model or observations) contain (known) errors that would
13-
normally prevent them from being processed. The issues can be in the
14-
metadata describing the dataset and/or in the actual data. Typical
15-
examples of such errors are missing or wrong attributes (e.g.
16-
attribute ''units'' says 1e-9 but data are actually in 1e-6), missing
17-
or mislabeled coordinates (e.g. ''lev'' instead of ''plev'' or missing
13+
normally prevent them from being processed. The issues can be in
14+
the metadata describing the dataset and/or in the actual data.
15+
Typical examples of such errors are missing or wrong attributes (e.g.
16+
attribute ''units'' says 1e-9 but data are actually in 1e-6), missing or
17+
mislabeled coordinates (e.g. ''lev'' instead of ''plev'' or missing
1818
coordinate bounds like ''lat_bnds'') or problems with the actual data
19-
(e.g. cloud liquid water only instead of sum of liquid + ice as
20-
specified by the CMIP data request).
19+
(e.g. cloud liquid water only instead of sum of liquid + ice as specified by the CMIP data request).
2120

2221
As an extreme case, some others data sources simply are not NetCDF
2322
files and must go through other data load function.
2423

2524
The ESMValCore can apply on the fly fixes to such datasets when
26-
issues can be fixed automatically. This is implemented for a set
27-
of `Natively supported non-CMIP datasets`_. The following provide
25+
issues can be fixed automatically. This is implemented for a set
26+
of `Natively supported non-CMIP datasets`_. The following provide
2827
details on how to design such fixes.
2928

3029
.. note::
31-
32-
**CMORizer scripts**. Support for many observational and reanalysis
33-
datasets is also possible through a priori reformating by
30+
31+
**CMORizer scripts**. Support for many observational and reanalysis
32+
datasets is also possible through a priori reformatting by
3433
:ref:`CMORizer scripts in the ESMValTool <esmvaltool:new-dataset>`,
3534
which are rather relevant for datasets of small volume
3635

3736
.. _fix_structure:
38-
37+
3938
Fix structure
4039
=============
4140

@@ -337,9 +336,38 @@ Natively supported non-CMIP datasets
337336
Some fixed datasets and native models formats are supported through
338337
the ``native6`` project or through a dedicated project.
339338
339+
ERA5 and MSWEP datasets
340+
-----------------------
341+
Put the files containing the data in the directory that you have configured
342+
for the ``native6`` project in your :ref:`user configuration file`, in a
343+
subdirectory called ``Tier{tier}/{dataset}/{version}/{frequency}/{short_name}``.
344+
Replace the items in curly braces by the values used in the variable/dataset
345+
definition in the :ref:`recipe <recipe_overview>`.
346+
Below is a list of datasets currently supported.
347+
348+
ERA5
349+
~~~~
350+
351+
- Supported variables: ``clt``, ``evspsbl``, ``evspsblpot``, ``mrro``, ``pr``, ``prsn``, ``ps``, ``psl``, ``ptype``, ``rls``, ``rlds``, ``rsds``, ``rsdt``, ``rss``, ``uas``, ``vas``, ``tas``, ``tasmax``, ``tasmin``, ``tdps``, ``ts``, ``tsn`` (``E1hr``/``Amon``), ``orog`` (``fx``)
352+
- Tier: 3
353+
354+
MSWEP
355+
~~~~~
356+
357+
- Supported variables: ``pr``
358+
- Supported frequencies: ``mon``, ``day``, ``3hr``.
359+
- Tier: 3
360+
361+
For example for monthly data, place the files in the ``/Tier3/MSWEP/latestversion/mon/pr`` subdirectory of your ``native6`` project location.
362+
363+
.. note::
364+
For monthly data (V220), the data must be postfixed with the date, i.e. rename ``global_monthly_050deg.nc`` to ``global_monthly_050deg_197901-201710.nc``
365+
366+
For more info: http://www.gloh2o.org/
367+
340368
.. _fixing_native_models:
341369
342-
Native models : IPSL-CM6,...
370+
Native models : IPSL-CM6,...
343371
-----------------------------
344372
345373
The following models are natively supported through the procedure
@@ -355,13 +383,13 @@ described above (:ref:`fix_structure`) and at
355383
datasets:
356384
- {simulation: CM61-LR-hist-03.1950, exp: piControl, freq: Analyse/TS_MO,
357385
account: p86caub, status: PROD, dataset: IPSL-CM6, project: IPSLCM,
358-
root: /thredds/tgcc/store}
386+
root: /thredds/tgcc/store}
359387
- {simulation: CM61-LR-hist-03.1950, exp: historical, freq: Output/MO,
360388
account: p86caub, status: PROD, dataset: IPSL-CM6, project: IPSLCM,
361-
root: /thredds/tgcc/store}
389+
root: /thredds/tgcc/store}
362390
363391
.. _ipslcm_extra_facets_example:
364-
392+
365393
The ``Output`` format is an example of a case where variables are
366394
grouped in multi-variable files, which name cannot be computed
367395
directly from datasets attributes alone but requires to use an
@@ -371,37 +399,6 @@ described above (:ref:`fix_structure`) and at
371399
multi-variable files must also undergo some data selection, which
372400
may involve an external process for performance purpose.
373401
374-
375-
376-
ERA5 and MSWEP datasets
377-
-----------------------
378-
Put the files containing the data in the
379-
directory that you have configured for the ``native6`` project in your
380-
:ref:`user configuration file`, in a subdirectory called
381-
``Tier{tier}/{dataset}/{version}/{frequency}/{short_name}``. Replace
382-
the items in curly braces by the values used in the variable/dataset
383-
definition in the :ref:`recipe <recipe_overview>`. Below is a list of
384-
datasets currently supported :
385-
386-
- **ERA5**
387-
388-
- Supported variables: ``clt``, ``evspsbl``, ``evspsblpot``, ``mrro``, ``pr``, ``prsn``, ``ps``, ``psl``, ``ptype``, ``rls``, ``rlds``, ``rsds``, ``rsdt``, ``rss``, ``uas``, ``vas``, ``tas``, ``tasmax``, ``tasmin``, ``tdps``, ``ts``, ``tsn`` (``E1hr``/``Amon``), ``orog`` (``fx``)
389-
- Tier: 3
390-
391-
- **MSWEP**
392-
393-
- Supported variables: ``pr``
394-
- Supported frequencies: ``mon``, ``day``, ``3hr``.
395-
- Tier: 3
396-
397-
For example for monthly data, place the files in the ``/Tier3/MSWEP/latestversion/mon/pr`` subdirectory of your ``native6`` project location.
398-
399-
.. note::
400-
401-
For monthly data (V220), the data must be postfixed with the date, i.e. rename ``global_monthly_050deg.nc`` to ``global_monthly_050deg_197901-201710.nc``
402-
403-
For more info: http://www.gloh2o.org/
404-
405402
.. _extra-facets-fixes:
406403
407404
Use of extra facets in fixes

doc/quickstart/configure.rst

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -176,8 +176,8 @@ It will be installed along with ESMValCore and can also be viewed on GitHub:
176176
`esmvalcore/config-developer.yml
177177
<https://github.com/ESMValGroup/ESMValCore/blob/main/esmvalcore/config-developer.yml>`_.
178178
This configuration file describes the file system structure and CMOR tables for several
179-
key projects (CMIP6, CMIP5, obs4mips, OBS6, OBS) on several key machines (e.g.
180-
BADC, CP4CDS, DKRZ, ETHZ, SMHI, BSC), and for native output data for some
179+
key projects (CMIP6, CMIP5, obs4mips, OBS6, OBS) on several key machines (e.g. BADC, CP4CDS, DKRZ,
180+
ETHZ, SMHI, BSC), and for native output data for some
181181
models (IPSL, ... see :ref:`configure_native_models`).
182182
CMIP data is stored as part of the Earth System Grid
183183
Federation (ESGF) and the standards for file naming and paths to files are set
@@ -262,10 +262,9 @@ your data please see :ref:`CMOR-DRS`.
262262
Preprocessor output files
263263
-------------------------
264264

265-
The filename to use for preprocessed data is configured in a similar
266-
manner using ``output_file``. Note that the extension ``.nc`` (and if
267-
applicable, a start and end time) will automatically be appended to
268-
the filename.
265+
The filename to use for preprocessed data is configured in a similar manner
266+
using ``output_file``. Note that the extension ``.nc`` (and if applicable,
267+
a start and end time) will automatically be appended to the filename.
269268

270269
.. _cmor_table_configuration:
271270

@@ -293,13 +292,13 @@ related to CMOR table settings available:
293292
Defaults to the value provided in ``cmor_type``.
294293

295294
.. _configure_native_models:
296-
295+
297296
Configuring native models and observation data sets
298297
----------------------------------------------------
299298

300299
ESMValCore can be configured for handling native model output formats
301300
and specific
302-
observation data sets without preliminary reformating. You can choose
301+
observation data sets without preliminary reformatting. You can choose
303302
to host this new data source either under a dedicated project or under
304303
project ``native6``; when choosing the latter, such a configuration
305304
involves the following steps:
@@ -314,9 +313,9 @@ involves the following steps:
314313

315314
.. code-block:: yaml
316315
317-
native6:
318-
...
319-
input_dir:
316+
native6:
317+
...
318+
input_dir:
320319
default: 'Tier{tier}/{dataset}/{latestversion}/{frequency}/{short_name}'
321320
MY_DATA_ORG: '{model}/{exp}/{simulation}/{version}/{type}'
322321
input_file:

doc/quickstart/find_data.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ section.
5050

5151
Native model data
5252
---------------------------------------------------------
53-
Support for native model data that is not formatted according to a CMIP
53+
Support for native model data that is not formatted according to a CMIP
5454
data request is quite easy using basic
5555
:ref:`ESMValCore fix procedure <fixing_data>` and has been implemented
5656
for some models :ref:`as described here <fixing_native_models>`

0 commit comments

Comments
 (0)