Skip to content

Commit 9e664ab

Browse files
authored
instructions to add/replace data files (#17)
* instructions to add/replace data files * add release notes * rename the release notes file --------- Signed-off-by: Jose Borreguero <[email protected]>
1 parent c078ea8 commit 9e664ab

File tree

3 files changed

+64
-4
lines changed

3 files changed

+64
-4
lines changed

docs/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ User Guide
1111

1212
/source/user/conda_environments
1313
/source/user/reduce
14-
/source/releases
14+
/source/release_notes
1515

1616
Contacting the Team
1717
+++++++++++++++++++

docs/source/developer/developer.rst

Lines changed: 62 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,10 @@ synchronized across these files:
5555

5656
Using the Data Repository usansred-data
5757
---------------------------------------
58+
Below follows a simplified rendering of the more detailed instructions on
59+
`how to use a data repository <https://ornl-neutrons.atlassian.net/wiki/spaces/NDPD/pages/19103745/Using+git-lfs+for+test+data>`_
60+
(login required).
61+
5862
To run the integration tests in your local environment, it is necessary first to download the data files.
5963
Because of their size, the files are stored in the Git LFS repository
6064
`usansred-data <https://code.ornl.gov/sns-hfir-scse/infrastructure/test-data/usansred-data>`_.
@@ -75,9 +79,64 @@ After this step, initialize or update the data repository:
7579
This will either clone `usansred-data` into `/path/to/usanred/tests/usansred-data` or
7680
bring the `usansred-data`'s refspec in sync with the refspec listed within file `/path/to/usanred/.gitmodules`.
7781

78-
An intro to Git LFS in the context of the Neutron Data Project is found in the
79-
`Confluence pages <https://ornl-neutrons.atlassian.net/wiki/spaces/NDPD/pages/19103745/Using+git-lfs+for+test+data>`_
80-
(login required).
82+
You can make sure all the large files are downloaded by issuing a pull command:
83+
84+
.. code-block:: bash
85+
86+
$> cd tests/usansred-data
87+
$> git checkout main
88+
$> git lfs pull
89+
90+
Replacing or adding a file in the data repository
91+
+++++++++++++++++++++++++++++++++++++++++++++++++
92+
When adding a new test that requires a new data file, or when replacing an existing data file because of
93+
some update to an existing test.
94+
95+
As an example, consider the data files used in
96+
`tests/usansred/test_reduce.py::test_main() <https://github.com/neutrons/usansred/blob/next/tests/usansred/test_reduce.py>`_:
97+
98+
.. code-block:: bash
99+
100+
tests/usansred-data/IPTS-30410/shared/reduced/UN_EmptyPCell_det_1.txt
101+
tests/usansred-data/IPTS-30410/shared/reduced/UN_EmptyPCell_det_1_lb.txt
102+
tests/usansred-data/IPTS-30410/shared/reduced/UN_EmptyPCell_det_1_unscaled.txt
103+
...
104+
105+
If we wish to replace any of those files with new versions, we simple overwrite the file(s) in the data repository
106+
with the new versions. For the particular case of `test_reduce.py::test_main()`, the new files would be
107+
located inside the temporary directory `tmp_path` that is automatically created when the test runs.
108+
109+
.. code-block:: bash
110+
111+
$> cp /path/to/new/UN_EmptyPCell_det_1.txt tests/usansred-data/IPTS-30410/shared/reduced/UN_EmptyPCell_det_1.txt
112+
$> cp /path/to/new/UN_EmptyPCell_det_1_lb.txt tests/usansred-data/IPTS-30410/shared/reduced/UN_EmptyPCell_det_1_lb.txt
113+
114+
Assuming we only wish to modify these two files, the next step is to commit these changes in our local repository
115+
*usansred-data* and then push the changes to the remote repository.
116+
117+
.. code-block:: bash
118+
119+
$> cd /path/to/usansred/tests/usansred-data
120+
$> git add IPTS-30410/shared/reduced/UN_EmptyPCell_det_1.txt
121+
$> git add IPTS-30410/shared/reduced/UN_EmptyPCell_det_1_lb.txt
122+
$> git commit -m "Updated data files for test_reduce.py::test_main()"
123+
$> git push origin
124+
125+
We're not done yet! The next step is to update the submodule reference in our local *usansred* repository
126+
and then push the changes to the remote repository.
127+
128+
.. code-block:: bash
129+
130+
$> cd /path/to/usansred
131+
$> git add tests/usansred-data
132+
$> git commit -m "Updated usansred-data submodule"
133+
$> git push origin
134+
135+
That's it.
136+
The new data files are now available to all developers who pull the latest changes from the remote repository.
137+
138+
If we want to add a new data file instead of replacing one, the procedure is the same only we won't be
139+
overwriting any existing files.
81140

82141

83142
Coverage reports

docs/source/releases.rst renamed to docs/source/release_notes.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Release Notes
1313

1414
**Of interest to the Developer:**
1515

16+
- PR 17: instructions to add/replace data files
1617
- PR 15: update Mantid dependency to 6.11
1718
- PR 14: transition from pip to conda when installing dependency finddata
1819
- PR 13: Take average of intensity values with duplicate Q AND solve the issue with bg interpolation when bg q and sample q values are too close or identical

0 commit comments

Comments
 (0)