Merge pull request #27 from scipp/orso-filewriter

Reimplement ORSO filewriter
scipp · Feb 7, 2024 · d566404 · d566404
2 parents b764f28 + a0dbe24
commit d566404
Show file tree

Hide file tree

Showing 29 changed files with 606 additions and 250 deletions.
diff --git a/conda/meta.yaml b/conda/meta.yaml
@@ -12,6 +12,7 @@ requirements:
     - setuptools_scm
   run:
     - dask
+    - python-dateutil
     - graphviz
     - plopp
     - pythreejs

diff --git a/docs/api-reference/index.md b/docs/api-reference/index.md
@@ -29,7 +29,8 @@
    :template: module-template.rst
    :recursive:
 
-   types
    amor
+   orso
    supermirror
+   types
 ```
diff --git a/docs/examples/amor.ipynb b/docs/examples/amor.ipynb
@@ -29,7 +29,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "params={\n",
+    "params = {\n",
     "    **default_parameters,\n",
     "    QBins: sc.geomspace(dim='Q', start=0.008, stop=0.075, num=200, unit='1/angstrom'),\n",
     "    SampleRotation[Sample]: sc.scalar(0.7989, unit='deg'),\n",
@@ -118,6 +118,150 @@
    "source": [
     "This plot can be used to check if the value of the sample rotation angle $\\omega$ is correct. The bright triangles should be pointing back to the origin $\\lambda = \\theta = 0$."
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Save data\n",
+    "\n",
+    "We can save the computed $I(Q)$ to an [ORSO](https://www.reflectometry.org) [.ort](https://github.com/reflectivity/file_format/blob/master/specification.md) file using the [orsopy](https://orsopy.readthedocs.io/en/latest/index.html) package.\n",
+    "\n",
+    "First, we need to collect the metadata for that file.\n",
+    "To this end, we build a pipeline with additional providers.\n",
+    "We also insert a parameter to indicate the creator of the processed data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from essreflectometry import orso\n",
+    "from essreflectometry.amor import orso as amor_orso\n",
+    "from orsopy import fileio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "providers_with_metadata = (\n",
+    "    *providers,\n",
+    "    *orso.providers,\n",
+    "    *amor_orso.providers,\n",
+    ")\n",
+    "\n",
+    "params[orso.OrsoCreator] = orso.OrsoCreator(fileio.base.Person(\n",
+    "    name='Max Mustermann',\n",
+    "    affiliation='European Spallation Source ERIC',\n",
+    "    contact='[email protected]',\n",
+    "))\n",
+    "\n",
+    "metadata_pipeline = sciline.Pipeline(\n",
+    "    providers_with_metadata,\n",
+    "    params=params\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then, we recompute $I(Q)$ and and combine it with the ORSO metadata:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "iofq_dataset = metadata_pipeline.compute(orso.OrsoIofQDataset)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Unfortunately, some metadata could not be determined autoamtically.\n",
+    "In particular, we need to specify the sample manually:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "iofq_dataset.info.data_source.sample"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "iofq_dataset.info.data_source.sample = fileio.data_source.Sample(\n",
+    "    name='Ni / Ti Multilayer',\n",
+    "    model=fileio.data_source.SampleModel(\n",
+    "      stack='air | (Ni | Ti) * 5 | Si',\n",
+    "    ),\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "And we also add the URL of this notebook to make it easier to reproduce the data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "iofq_dataset.info.reduction.script = 'https://scipp.github.io/essreflectometry/examples/amor.html'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Finally, we can save the data to a file.\n",
+    "Note that `iofq_dataset` is an [orsopy.fileio.orso.OrsoDataset](https://orsopy.readthedocs.io/en/latest/orsopy.fileio.orso.html#orsopy.fileio.orso.OrsoDataset)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "iofq_dataset.save('amor_reduced_iofq.ort')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Look at the first 50 lines of the file to inspect the metadata:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!head amor_reduced_iofq.ort -n50"
+   ]
   }
  ],
  "metadata": {
@@ -136,7 +280,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,

diff --git a/pyproject.toml b/pyproject.toml
@@ -31,6 +31,7 @@ requires-python = ">=3.10"
 # Make sure to list one dependency per line.
 dependencies = [
   "dask",
+  "python-dateutil",
   "graphviz",
   "plopp",
   "pythreejs",

diff --git a/requirements/base.in b/requirements/base.in
@@ -6,6 +6,7 @@ ipython==8.9.0
 # --- END OF CUSTOM SECTION ---
 # The following was generated by 'tox -e deps', DO NOT EDIT MANUALLY!
 dask
+python-dateutil
 graphviz
 plopp
 pythreejs

diff --git a/requirements/base.txt b/requirements/base.txt
@@ -1,4 +1,4 @@
-# SHA1:301651c34ea6fe2705e74c10ff13dedb897c14ff
+# SHA1:df5a1bd38eaeab3b3099d7d737171e2192615a39
 #
 # This file is autogenerated by pip-compile-multi
 # To update, run:
@@ -9,7 +9,7 @@ asttokens==2.4.1
     # via stack-data
 backcall==0.2.0
     # via ipython
-certifi==2023.11.17
+certifi==2024.2.2
     # via requests
 charset-normalizer==3.3.2
     # via requests
@@ -23,13 +23,13 @@ contourpy==1.2.0
     # via matplotlib
 cycler==0.12.1
     # via matplotlib
-dask==2023.12.1
+dask==2024.1.1
     # via -r base.in
 decorator==5.1.1
     # via ipython
 executing==2.0.1
     # via stack-data
-fonttools==4.47.0
+fonttools==4.47.2
     # via matplotlib
 fsspec==2023.12.2
     # via dask
@@ -92,9 +92,9 @@ pickleshare==0.7.5
     # via ipython
 pillow==10.2.0
     # via matplotlib
-platformdirs==4.1.0
+platformdirs==4.2.0
     # via pooch
-plopp==23.11.0
+plopp==24.1.1
     # via -r base.in
 pooch==1.8.0
     # via scippneutron
@@ -112,6 +112,7 @@ pyparsing==3.1.1
     # via matplotlib
 python-dateutil==2.8.2
     # via
+    #   -r base.in
     #   matplotlib
     #   scippnexus
 pythreejs==2.4.2
@@ -122,18 +123,18 @@ pyyaml==6.0.1
     #   orsopy
 requests==2.31.0
     # via pooch
-sciline==24.1.0
+sciline==24.1.1
     # via -r base.in
 scipp==23.12.0
     # via
     #   -r base.in
     #   scippneutron
     #   scippnexus
-scippneutron==23.11.0
+scippneutron==24.1.0
     # via -r base.in
-scippnexus==23.12.0
+scippnexus==23.12.1
     # via scippneutron
-scipy==1.11.4
+scipy==1.12.0
     # via
     #   scippneutron
     #   scippnexus
@@ -143,7 +144,7 @@ six==1.16.0
     #   python-dateutil
 stack-data==0.6.3
     # via ipython
-toolz==0.12.0
+toolz==0.12.1
     # via
     #   dask
     #   partd
@@ -157,9 +158,9 @@ traitlets==5.14.1
     #   traittypes
 traittypes==0.2.1
     # via ipydatawidgets
-urllib3==2.1.0
+urllib3==2.2.0
     # via requests
-wcwidth==0.2.12
+wcwidth==0.2.13
     # via prompt-toolkit
 widgetsnbextension==4.0.9
     # via ipywidgets

diff --git a/requirements/basetest.txt b/requirements/basetest.txt
@@ -11,9 +11,9 @@ iniconfig==2.0.0
     # via pytest
 packaging==23.2
     # via pytest
-pluggy==1.3.0
+pluggy==1.4.0
     # via pytest
-pytest==7.4.4
+pytest==8.0.0
     # via -r basetest.in
 tomli==2.0.1
     # via pytest
diff --git a/requirements/ci.txt b/requirements/ci.txt
@@ -7,7 +7,7 @@
 #
 cachetools==5.3.2
     # via tox
-certifi==2023.11.17
+certifi==2024.2.2
     # via requests
 chardet==5.2.0
     # via tox
@@ -23,7 +23,7 @@ filelock==3.13.1
     #   virtualenv
 gitdb==4.0.11
     # via gitpython
-gitpython==3.1.40
+gitpython==3.1.41
     # via -r ci.in
 idna==3.6
     # via requests
@@ -32,11 +32,11 @@ packaging==23.2
     #   -r ci.in
     #   pyproject-api
     #   tox
-platformdirs==4.1.0
+platformdirs==4.2.0
     # via
     #   tox
     #   virtualenv
-pluggy==1.3.0
+pluggy==1.4.0
     # via tox
 pyproject-api==1.6.1
     # via tox
@@ -48,9 +48,9 @@ tomli==2.0.1
     # via
     #   pyproject-api
     #   tox
-tox==4.11.4
+tox==4.12.1
     # via -r ci.in
-urllib3==2.1.0
+urllib3==2.2.0
     # via requests
 virtualenv==20.25.0
     # via tox