Skip to content

Latest commit

 

History

History
249 lines (150 loc) · 8.65 KB

README_developer.rst

File metadata and controls

249 lines (150 loc) · 8.65 KB

drt-sans

Data Reduction Toolkit for Small Angle Neutron Scattering

This packages is a collection of functionality for reducing SANS data developed in collaboration with the instrument scientists at the High Flux Isotope Reactor (HFIR) and Spallation Neutron Source (SNS) at Oak Ridge National Laboratory. While much of the functionality is generic, this implementation is aimed at reducing data from BIOSANS, EQSANS, and GPSANS. As appropriate, this work is an abstraction layer on top of the mantid project.

This is a python3 only package.

Set-up for development in a virtual environment

These are the instructions for a person who is developing both drt-sans and mantid in tandem. It assumes that you are using a virtual environment and have a local build of mantid. As one caveat, for this method, mantid must be build against the same version of python being used in the virtual environment

  1. Checkout the code.
$ git clone [email protected]:sns-hfir-scse/sans/sans-backend.git
$ cd sans-backend
  1. Create the development environment.

Create a conda environment and activate it.

  • It will use Mantid’s conda package.
  • miniconda/conda is required.
  • Linux command: Use the libmamba solver for faster install
$ conda config --add channels conda-forge
$ conda config --add channels mantid
$ conda env create # will use environment.yml
$ conda activate drtsans-dev

Warning: if you create the conda environment on SNS’s analysis cluster, avoid naming your environment as ‘drtsans’, ‘drtsans-qa’ and ‘drtsans-dev’, which are reserved.

To make it easier to use, configure direnv to activate the conda environment when you enter the directory.

  1. Install the code in develop mode.
$ pip install -e .
  1. Try it out. Start python and try
import mantid
import drtsans

Verify you can run the unit tests:

$ python -m pytest tests/unit/
$ python -m pytest tests/integration/

Some unit and integration tests require testing data. Thus these tests can be either run on * SNS analysis cluster or * Local computer with mount to /SNS/ archive. Here is the instruction to mount SNS data directories.

  1. When done, deactivate the virtual environment using
$ deactivate

for virtual environment. Or

$ deactivate

for conda environment.

Git submodule for test data

To get the current version of the git-lfs associated with the currently checked out branch

$ git submodule update --init

where update checks out the associated refspec and --init tells git to initialize the submodule if it isn't already. To update the submodule to the latest commit on the branch being tracked

$ git submodule update --remote --merge

where --remote tells git to fetch the latest changes from the upstream repository and --merge tells git to merge the changes into the working tree.

Then, to get the data files associated with the submodule, run

$ cd tests/data/drtsans-data
$ git lfs fetch

See the git-submodules documentation for more detailed information.

Running the tests

The tests for this project are all written using pytest. The build pipeline currently `runs the unit tests and integration tests separately using

$ python -m pytest tests/unit/
$ python -m pytest tests/integration/

This is one of the ways pytest allows for selecting tests. Specifying a directory or file will run all tests within that directory (recursively) or file. Specifying a regular expression using -k will select all tests that match the regular expression independent of where they are defined

$ python -m pytest -k test_samplelogs

To run an individual test within an individual file add :: to the filename to specify the test

$ python -m pytest tests/unit/new/drtsans/tof/eqsans/test_beam_finder.py::test_center_detector

Building the documentation

The site can be built directly using

$ sphinx-build -b html docs/ build/sphinx/html

or

$ python setup.py build_sphinx

Installing the pre-commit hook

To automatically run the pre-commit steps (e.g. linting) when adding a commit, install the git pre-commit hook.

$ pre-commit install

To run pre-commit locally without committing

$ pre-commit run --all

Contributing

Contributing is done through merge requests of code that you have the permission to add. See CONTRIBUTING.rst for more information.

Test Driven Development (TDD)

  • Test driven Development

    drtSANS development follows test-driven development (TDD) process [1]. All software requirements for SANS data reduction shall be converted to test cases before software is fully developed. All software developments are tracked by repeatedly testing the software against all test cases.

  • Unit test

    All methods and modules shall have unit tests implemented. Unit tests are located in repo/tests/unit/new. A unit test shall be created in the corresponding directory to the method or module that it tests against.

    Examples:

  • Integration test

    Integration test will test the combination of Individual modules and methods. Integration tests can be

    • general for all instrument, for instance tests/integration/new/drtsans/test_stitch.py.
    • specific to a suite of similar instruments, for instance tests/integration/new/drtsans/mono/test_transmission.py for all mono-wavelength instruments including Bio-SANS and GP-SANS.
    • specific to an individual instrument, for instance, tests/integration/new/drtsans/mono/gpsans/test_find_beam_center.py for GP-SANS and tests/integration/new/drtsans/tof/eqsans/test_apply_solid_angle.py for EQ-SANS.

  • Testing data location

    Testing data are located on SNS data archive: /SNS/EQSANS/shared/sans-backend/data.

    Testing data for specific instruments have specific locations:

    • EQSANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/sns/eqsans/
    • Bio-SANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/hfir/biosans/
    • GP-SANS: /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/hfir/gpsans/

    Data files are referenced in the tests via the reference_dir pytest fixture. For instance, reference_dir.new.eqsans points to /SNS/EQSANS/shared/sans-backend/data/new/ornl/sans/sns/eqsans/

Required libraries