Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nexus transforms improvements #126

Merged
merged 11 commits into from
Nov 5, 2024
Merged

Nexus transforms improvements #126

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e53857c
Expose GenericNeXusWorkflow in nexus module
SimonHeybrock Nov 4, 2024
67c7c93
Rename PulseSelection -> TimeInterval
SimonHeybrock Nov 4, 2024
49afdd8
Check earlier for bad time dependence
SimonHeybrock Nov 4, 2024
6619f57
Rename function
SimonHeybrock Nov 4, 2024
97d0ec5
Add mechanism for processing time-dependent transformation chains
SimonHeybrock Nov 4, 2024
cddfcdc
Remove dodgy and likely insufficient time-filter mechanism
SimonHeybrock Nov 4, 2024
dbc2b9c
Match input dim order and offset units
SimonHeybrock Nov 4, 2024
70d9443
Remove tests written during feature development
SimonHeybrock Nov 4, 2024
853544d
Bump scippnexus
SimonHeybrock Nov 5, 2024
63a7d45
spelling
SimonHeybrock Nov 5, 2024
0ce71d5
Explain choice of final bin size
SimonHeybrock Nov 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ requires-python = ">=3.10"
dependencies = [
"sciline>=24.06.2",
"scipp>=24.02.0",
"scippnexus>=24.10.0",
"scippnexus>=24.11.0",
]

dynamic = ["version"]
Expand Down
2 changes: 1 addition & 1 deletion requirements/base.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
# The following was generated by 'tox -e deps', DO NOT EDIT MANUALLY!
sciline>=24.06.2
scipp>=24.02.0
scippnexus>=24.10.0
scippnexus>=24.11.0
10 changes: 5 additions & 5 deletions requirements/base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# SHA1:f3f405557d50cce037f6d681dfb1c5c13f5bef49
# SHA1:61ced0b7e1301f66609a2d72adaad2ad2e729000
#
# This file is autogenerated by pip-compile-multi
# To update, run:
#
# pip-compile-multi
#
cyclebane==24.6.0
cyclebane==24.10.0
# via sciline
h5py==3.12.1
# via scippnexus
networkx==3.3
networkx==3.4.2
# via cyclebane
numpy==2.1.2
numpy==2.1.3
# via
# h5py
# scipp
Expand All @@ -24,7 +24,7 @@ scipp==24.9.1
# via
# -r base.in
# scippnexus
scippnexus==24.10.0
scippnexus==24.11.0
# via -r base.in
scipy==1.14.1
# via scippnexus
Expand Down
4 changes: 2 additions & 2 deletions requirements/basetest.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ asttokens==2.4.1
# via stack-data
certifi==2024.8.30
# via requests
charset-normalizer==3.3.2
charset-normalizer==3.4.0
# via requests
comm==0.2.2
# via ipywidgets
Expand All @@ -25,7 +25,7 @@ idna==3.10
# via requests
iniconfig==2.0.0
# via pytest
ipython==8.28.0
ipython==8.29.0
# via ipywidgets
ipywidgets==8.1.5
# via -r basetest.in
Expand Down
8 changes: 4 additions & 4 deletions requirements/ci.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ certifi==2024.8.30
# via requests
chardet==5.2.0
# via tox
charset-normalizer==3.3.2
charset-normalizer==3.4.0
# via requests
colorama==0.4.6
# via tox
distlib==0.3.8
distlib==0.3.9
# via virtualenv
filelock==3.16.1
# via
Expand Down Expand Up @@ -48,11 +48,11 @@ tomli==2.0.2
# via
# pyproject-api
# tox
tox==4.21.2
tox==4.23.2
# via -r ci.in
typing-extensions==4.12.2
# via tox
urllib3==2.2.3
# via requests
virtualenv==20.26.6
virtualenv==20.27.1
# via tox
6 changes: 3 additions & 3 deletions requirements/dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
-r wheels.txt
annotated-types==0.7.0
# via pydantic
anyio==4.6.0
anyio==4.6.2.post1
# via
# httpx
# jupyter-server
Expand All @@ -32,7 +32,7 @@ click==8.1.7
# via
# pip-compile-multi
# pip-tools
copier==9.3.1
copier==9.4.1
# via -r dev.in
dunamai==1.22.0
# via copier
Expand Down Expand Up @@ -71,7 +71,7 @@ jupyter-server==2.14.2
# notebook-shim
jupyter-server-terminals==0.5.3
# via jupyter-server
jupyterlab==4.2.5
jupyterlab==4.3.0
# via -r dev.in
jupyterlab-server==2.27.3
# via jupyterlab
Expand Down
25 changes: 12 additions & 13 deletions requirements/docs.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,17 @@ beautifulsoup4==4.12.3
# via
# nbconvert
# pydata-sphinx-theme
bleach==6.1.0
bleach==6.2.0
# via nbconvert
certifi==2024.8.30
# via requests
charset-normalizer==3.3.2
charset-normalizer==3.4.0
# via requests
comm==0.2.2
# via
# ipykernel
# ipywidgets
debugpy==1.8.6
debugpy==1.8.7
# via ipykernel
decorator==5.1.1
# via ipython
Expand All @@ -60,7 +60,7 @@ imagesize==1.4.1
# via sphinx
ipykernel==6.29.5
# via -r docs.in
ipython==8.28.0
ipython==8.29.0
# via
# -r docs.in
# ipykernel
Expand All @@ -77,7 +77,7 @@ jinja2==3.1.4
# sphinx
jsonschema==4.23.0
# via nbformat
jsonschema-specifications==2023.12.1
jsonschema-specifications==2024.10.1
# via jsonschema
jupyter-client==8.6.3
# via
Expand All @@ -98,7 +98,7 @@ markdown-it-py==3.0.0
# via
# mdit-py-plugins
# myst-parser
markupsafe==2.1.5
markupsafe==3.0.2
# via
# jinja2
# nbconvert
Expand Down Expand Up @@ -131,7 +131,6 @@ packaging==24.1
# via
# ipykernel
# nbconvert
# pydata-sphinx-theme
# sphinx
pandocfilters==1.5.1
# via nbconvert
Expand All @@ -143,13 +142,13 @@ platformdirs==4.3.6
# via jupyter-core
prompt-toolkit==3.0.48
# via ipython
psutil==6.0.0
psutil==6.1.0
# via ipykernel
ptyprocess==0.7.0
# via pexpect
pure-eval==0.2.3
# via stack-data
pydata-sphinx-theme==0.15.4
pydata-sphinx-theme==0.16.0
# via -r docs.in
pygments==2.18.0
# via
Expand All @@ -170,15 +169,15 @@ referencing==0.35.1
# jsonschema-specifications
requests==2.32.3
# via sphinx
rpds-py==0.20.0
rpds-py==0.20.1
# via
# jsonschema
# referencing
snowballstemmer==2.2.0
# via sphinx
soupsieve==2.6
# via beautifulsoup4
sphinx==8.0.2
sphinx==8.1.3
# via
# -r docs.in
# myst-parser
Expand All @@ -187,7 +186,7 @@ sphinx==8.0.2
# sphinx-autodoc-typehints
# sphinx-copybutton
# sphinx-design
sphinx-autodoc-typehints==2.4.4
sphinx-autodoc-typehints==2.5.0
# via -r docs.in
sphinx-copybutton==0.5.2
# via -r docs.in
Expand All @@ -207,7 +206,7 @@ sphinxcontrib-serializinghtml==2.0.0
# via sphinx
stack-data==0.6.3
# via ipython
tinycss2==1.3.0
tinycss2==1.4.0
# via nbconvert
tomli==2.0.2
# via sphinx
Expand Down
2 changes: 1 addition & 1 deletion requirements/mypy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# pip-compile-multi
#
-r test.txt
mypy==1.11.2
mypy==1.13.0
# via -r mypy.in
mypy-extensions==1.0.0
# via mypy
4 changes: 2 additions & 2 deletions requirements/nightly.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ cyclebane @ git+https://github.com/scipp/cyclebane@main
# sciline
h5py==3.12.1
# via scippnexus
networkx==3.3
networkx==3.4.2
# via cyclebane
numpy==2.1.2
numpy==2.1.3
# via
# h5py
# scipp
Expand Down
6 changes: 3 additions & 3 deletions requirements/static.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#
cfgv==3.4.0
# via pre-commit
distlib==0.3.8
distlib==0.3.9
# via virtualenv
filelock==3.16.1
# via virtualenv
Expand All @@ -17,9 +17,9 @@ nodeenv==1.9.1
# via pre-commit
platformdirs==4.3.6
# via virtualenv
pre-commit==4.0.0
pre-commit==4.0.1
# via -r static.in
pyyaml==6.0.2
# via pre-commit
virtualenv==20.26.6
virtualenv==20.27.1
# via pre-commit
2 changes: 2 additions & 0 deletions src/ess/reduce/nexus/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"""

from . import types
from .workflow import GenericNeXusWorkflow
from ._nexus_loader import (
load_data,
group_event_data,
Expand All @@ -28,4 +29,5 @@
'load_component',
'compute_component_position',
'extract_signal_data_array',
'GenericNeXusWorkflow',
]
10 changes: 5 additions & 5 deletions src/ess/reduce/nexus/types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,10 +192,10 @@ class Filename(sciline.Scope[RunType, Path], Path): ...


@dataclass
class PulseSelection(Generic[RunType]):
class TimeInterval(Generic[RunType]):
"""Range of neutron pulses to load from NXevent_data or NXdata groups."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only pulses or also logs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the logs in NXtransformations it is loading the full log. The time interval is later used to determine which values are relevant.

The workflow is not loading any other logs currently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But is your intention to use TimeInterval for other logs, too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really at the moment, since the label-based slicing in Scipp and ScippNeXus does not do what we need (include the previous value). We thus want to load "more" than the naive slice says. Unless we get very large logs it seems easier to just load everything, and then move events to log values.


value: snx.typing.ScippIndex | slice
value: slice


@dataclass
Expand Down Expand Up @@ -253,7 +253,7 @@ def from_chain(
therefore currently raise an error if the transformation chain does not compute
to a scalar.
"""
if chain.transformations.sizes != {}:
raise ValueError(f"Expected scalar transformation, got {chain}")
transform = chain.compute()
if transform.ndim == 0:
return NeXusTransformation(value=transform)
raise ValueError(f"Expected scalar transformation, got {transform}")
return NeXusTransformation(value=transform)
Loading