Resolve Timestamp Issues #10
Conversation
|
|
||
| time_msg = (msg >> np.uint(16)) & np.uint(0xFFFF) | ||
| heartbeat_msb = time_msg << np.uint(32) | ||
| # TODO the c++ code has large jump detection, do not understand why |
There was a problem hiding this comment.
I suspect that this line is the actual bug that needed to be fixed.
There was a problem hiding this comment.
I'd like to see how the heartbeats are being stored in the actual data (so far I haven't seen any heartbeat messages). @JGoodrichBNL is this the right Run ID to check: 143210? Do you remember by any chance what heartbeat rate was set up in the detector when you acquired those data?
There was a problem hiding this comment.
Hi @genematx,
No, that scan is quite old.
Here are some scans with heartbeats. Note that the data is very sparse as we did this without beam and the only hits are background/cosmic rays. We will try with beam on Monday or Tuesday during data security time.
All scans below are 100 partitions at at 1.000 sec exposure time/partition with an 1.002 sec acquire time/partition (e.g., 1.000 sec of collection followed by 2 ms deadtime before starting next partition).
152384 - global_timestamp_interval = 500
152389 - global_timestamp_interval = 250
152389 - global_timestamp_interval = 0 (i.e., what all of our old data had)
152396 - global_timestamp_interval = 1
152397 - global_timestamp_interval = 0.1
152401 - global_timestamp_interval = 0.01
We believe the values are in ms, although the timestamp rate in the files does not seem to match this more. @josephhanrahan can elaborate. Perhaps you can upload some of your viewing codes on your branch, Joe?
Please note we are coming up on our beamtime very quickly, so we would like to get this resolved ASAP.
Thanks!
There was a problem hiding this comment.
Thanks, Justin! That looks useful; I'll try them later.
There was a problem hiding this comment.
There doesn't seem to be any photon events, at least in the first run. This is still useful (e.g. there are some "command" messages in there), but we'll need something more real.
There was a problem hiding this comment.
Hmm, @josephhanrahan looked at all of these. There are not many events, but there should be some. He was able to reconstruct the scan length with his codes and show that there times beyond the SPIDR roll over. Joe, when you have a chance, maybe make a thread on Slack about this?
There was a problem hiding this comment.
I didn't check them all; there might be some events in other files.
|
Thanks a lot, @tacaswell, for reviewing this! I've left out some pieces of the code mostly to get the simplest working example first. Some of them, like heartbeat messages, were not being used at all. Justin and Joe have recently found a way to configure the detector to emit those messages, so I will bring this back. Likewise, with registering unknown headers -- I think it's a good idea to keep track of them explicitly, even if they are not used. |
…dling of KDTree neighbor list, updated conversion functions, and removed file-finding logic specific to CHX's current ophyd object.
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
This PR addresses the issues raised in https://jira.nsls2.bnl.gov/browse/CS-480 and related to improper handling of timestamp rollovers.
There are two distinct problems mentioned in the linked Jira ticket -- occasional spikes in timestamp values and periodic timestamp rollover -- however, both of them appear to be caused by the same reason: the overflow and reset of a 16-bit SPIDR counter after ~27 sec of operation. The implemented solution accounts for this limitation by introducing a
spidr_epochvariable, which is incremented after each counter overflow and is taken into account when computing the timestamp.