Find a reference time for computing pulse_index

[nvaytet]

See #180 (comment) for context.

When doing pulse-skipping, We need a reliable way to define a reference start time of the run, so we can determine to which pulse in a stride the events are coming from.
The reference time needs to be fixed, in case the data comes in in chunks (cannot simply use the min event_time_offset for each new piece of data).
Some pulses might have no events, so pulses are not always consecutive.

Using a global reference like epoch may work, but it seems difficult to know only from that if the filewriter started recording in the middle of a frame, or at the start of a frame.

It seems the only way would be to look at the open/close times of the pulse-skipping chopper and determine pulse indices from that.
This sounds complicated as we would need to flag which is the pulse-skipping chopper, propagate the open-close times back to the source, and assign pulse indices. The open-close times would need to be in the global timing reference, probably using the chopper TDC time stamps...

[SimonHeybrock]

How about the run start time recorded in the NeXus file?

[SimonHeybrock]

Some pulses might have no events, so pulses are not always consecutive.

Note also that there may be multiple messages per pulse, all with the same event_time_zero.

[nvaytet]

From discussion with @jokasimr:

The main thing is that we identify a pulse time when the chopper system has the same exact state as the chopper system in the simulation. That can be done purely from reading the chopper logs, therefore we don't need to trace any neutrons do this.

When determining if the chopper systems states are equal we don't have to specify which chopper is the pulse-skipping chopper, we can just make sure that they all have the right state. If we do it that way we don't have to specify what chopper is "pulse-skipping"

So for example, we could do it this way:

1. Go through all the choppers in the simulation, note how many pulse times from the start time their first tdc time is. For all choppers except the pulse skipping choppers this number will be 0. For the pulse skipping choppers this number is less than the number of skipped pulses.
2. Go through the pulses in the measurement until you find a pulse time where: For each chopper, the number of pulse periods to the next tdc is equal to the corresponding number in the simulation.

So for step 1, you probably wouldn't really need to go through any timings of choppers, you could just do n_pulses_until_next_tdc = (1.0 / chopper.frequency) // pulse_period

And for step 2, it means we need to also stream the chopper logs in any live data processing.
In live data, it does seem that the chopper logs get streamed alongside the events, continuously, in a separate topic.

[nvaytet]

For now, we have used a workaround in #186 which makes a guess by trying with a small number of events.
This seems to work well, but it is not clear whether it works in all cases.
We may still need to look at chopper data in the long run.