Simulation refactor #586

christophwelling · 2023-10-24T21:56:59Z

This is the refactor of the simulation.py script:

Simulations are split up between classes that handle the simulation at station, event and shower level
Writing the output is handled by its own class
Passing information around using member variables is gone. Only exceptions are the methods set_event_group and set_event.
I'm going to write some documentation about how this is supposed to work once the documentation builds properly again.

…simulation_refactor

…ndidate shower was found

… in Veff unit test

sjoerd-bouma · 2023-10-25T17:59:32Z

@christophwelling Haven't had a proper look at this yet, but it's probably worth merging in the latest develop - the tests (including the documentation) should complete successfully on those.

…simulation_refactor

christophwelling · 2023-10-27T16:07:53Z

The tests seem to work, but the documentation fails because of some class for ARA.

sjoerd-bouma · 2023-10-30T10:00:24Z

I don't think the ARA import is what breaks the documentation, but I'll take full responsibility for the documentation script somehow still being unable to properly sort the error messages. These are errors that should actually be fixed:

/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of set_shower in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/channel_efield_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of set_shower in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/channel_efield_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of set_event_group in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of set_event_group in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station_per_shower in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station_per_shower in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: Unknown section Parameters: in the docstring of add_station_per_shower in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/output_writer_hdf5.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: potentially wrong underline length... 
Parameters 
----------- in 
Set information about the station that is currently simulated.
... in the docstring of set_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/shower_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: potentially wrong underline length... 
Parameters 
----------- in 
Set information about the station that is currently simulated.
... in the docstring of set_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/shower_simulator.py.
  warn(msg)
/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/site-packages/numpydoc/docscrape.py:460: UserWarning: potentially wrong underline length... 
Parameters 
----------- in 
Set information about the station that is currently simulated.
... in the docstring of set_station in /home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/shower_simulator.py.
  warn(msg)
WARNING: autodoc: failed to import module 'generate_unforced' from module 'NuRadioMC.EvtGen'; the following exception was raised:
No module named 'NuRadioMC.simulation.simulation_base'
/home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py:docstring of NuRadioMC.simulation.hardware_response_simulator.hardwareResponseSimulator.simulate_detector_response:7: ERROR: Unexpected indentation.
/home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py:docstring of NuRadioMC.simulation.hardware_response_simulator:4: WARNING: Block quote ends without a blank line; unexpected unindent.
/home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py:docstring of NuRadioMC.simulation.hardware_response_simulator.hardwareResponseSimulator:1: WARNING: Definition list ends without a blank line; unexpected unindent.
/home/runner/work/NuRadioMC/NuRadioMC/NuRadioMC/simulation/hardware_response_simulator.py:docstring of NuRadioMC.simulation.hardware_response_simulator.hardwareResponseSimulator.simulate_detector_response:16: ERROR: Unexpected indentation.

Looks like one missing module (simulation_base), and some sections named Parameters: instead of Parameters. The underlines should be the same length as the section headings.

fschlueter · 2023-11-22T14:16:26Z

@christophwelling, what I can already say is that the branch is not compatible with numpy==1.24 which removed all aliases for built-in data types. I think it makes sense to already fix it here. They were already labeled deprecated with 1.20
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

fschlueter · 2023-11-22T14:58:15Z

I understand that you probably wanted to first just refactor the code without changing the functionality. However, should we not also cut "loose ends"? E.g.,

        if default_detector_station is not None:
            logger.warning(
                'Deprecation warning: Passing the default detector station is deprecated. Default stations and default'
                'channel should be specified in the detector description directly.'
            )
            logger.status(f"Default detector station provided (station {default_detector_station}) -> Using generic detector")
            self._det = NuRadioReco.detector.generic_detector.GenericDetector(json_filename=self._detectorfile, default_station=default_detector_station,
                                                 default_channel=default_detector_channel, antenna_by_depth=False)
        else:

fschlueter · 2023-11-22T17:09:42Z

The new "subclasses" like hardwareResponseSimulator do not log any information, warnings, .... . Is that correct?

sjoerd-bouma · 2023-11-23T08:47:03Z

I understand that you probably wanted to first just refactor the code without changing the functionality. However, should we not also cut "loose ends"? E.g.,

        if default_detector_station is not None:
            logger.warning(
                'Deprecation warning: Passing the default detector station is deprecated. Default stations and default'
                'channel should be specified in the detector description directly.'
            )
            logger.status(f"Default detector station provided (station {default_detector_station}) -> Using generic detector")
            self._det = NuRadioReco.detector.generic_detector.GenericDetector(json_filename=self._detectorfile, default_station=default_detector_station,
                                                 default_channel=default_detector_channel, antenna_by_depth=False)
        else:

I believe we haven't yet had a full release with this deprecation warning included, so I think we should keep it for 2.2.0 and only fully deprecate it in a future version.

christophwelling added 30 commits November 15, 2022 13:15

Implements simulation_base class

48ab6a2

Fix import of pretty_time_delta

7548384

Adds simulation_detector class

940afa9

Fix units import for simulation_detector

b1ebdc7

Fix undefined station_id variable in simulation_detector

8777997

Adds simulation_emission class

bd21153

Fix candidate station bug

b788ccb

Fix imports for simulation_emission

999e3da

Adds class to handle io operations for simulation.py

ea1d8d7

Put calculation of station barycenter into its own function

fe8043c

Put calculation of particle weights into its own function

0fb9e5e

Merge branch 'develop' of https://github.com/nu-radio/NuRadioMC into …

dc19522

…simulation_refactor

Put writing of .nur file into its own function

dc42288

Add simulation_propagation module

553a8fd

Put calculation of polarization angles into their own function

4bb6a69

Fixes issue where candidate_station was set to False again after a ca…

71da365

…ndidate shower was found

Puts distance cut check between channel and shower into its own function

13abcff

Puts the distance cut check for the station into its own function

27e5059

Fix station distance cut

54b5959

Puts the raytracing per channel into its own function

ddfa55d

Always calculate vertex distances and shower sums

b316c20

Calculate cherenkov angle only once for all channels

b543f88

Remove return statement that caused bug

d59a64e

Fix pulser examples

8f810e1

Put creation of event structure for output into its own function

4391b30

Put shower simulation into its own function

496ba34

Put all of shower simulation into extra function

78bb47a

Removes obsolete example

fe0d436

Make dummy detector simulation use its own dummy station

662d7c0

Fix variable name

66ed293

christophwelling added 18 commits September 25, 2023 11:45

Get detector sampling rate from detector instead of simulation object…

f718e4a

… in Veff unit test

Removes obsolete simulation_input_output class

f205a4b

Remove more obsolete code

3539ef8

Make private class variables inaccessible from outside

08b036e

Make input attributes no longer directly accessible from outside

6c1fc52

Fix effective volume calculation

e532c3f

Implement class to keep track of simulation time

62aadaa

Removes unused parameter when writing trigger results to file

98a8fef

Fix per channel distance cut

66bda46

Fix shower ID list

eabf80e

Some code cleanup

73247c9

Clean up import

71f4eb2

Add docstrings for channel_efield_simulator

5bbf448

Add docstrings to hardware response simulation

7dd7e15

Fix writing of trigger information into output files

c79f1bc

Add doctrings to HDF5 output writer

228f714

Add docstrings

caa4c5c

Remove misplaced README

91694eb

christophwelling requested a review from cg-laser October 24, 2023 21:57

Merge branch 'develop' of https://github.com/nu-radio/NuRadioMC into …

6882da3

…simulation_refactor

christophwelling added 2 commits November 3, 2023 16:08

Fixes import for prettyTimeDelta

8a8abfb

Fix docstrings

b541b19

sjoerd-bouma mentioned this pull request Nov 24, 2023

Release v2.2.0 #595

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simulation refactor #586

Simulation refactor #586

christophwelling commented Oct 24, 2023

sjoerd-bouma commented Oct 25, 2023

christophwelling commented Oct 27, 2023

sjoerd-bouma commented Oct 30, 2023

fschlueter commented Nov 22, 2023 •

edited

fschlueter commented Nov 22, 2023

fschlueter commented Nov 22, 2023

sjoerd-bouma commented Nov 23, 2023

Simulation refactor #586

Are you sure you want to change the base?

Simulation refactor #586

Conversation

christophwelling commented Oct 24, 2023

sjoerd-bouma commented Oct 25, 2023

christophwelling commented Oct 27, 2023

sjoerd-bouma commented Oct 30, 2023

fschlueter commented Nov 22, 2023 • edited

fschlueter commented Nov 22, 2023

fschlueter commented Nov 22, 2023

sjoerd-bouma commented Nov 23, 2023

fschlueter commented Nov 22, 2023 •

edited