uproot.dask behavior for partially readable files

[alexander-held]

The current behavior of uproot.open and uproot.dask differs when dealing with files that are partially unreadable by uproot. In my concrete example, I am dealing with ATLAS PHYSLITE files. The following snippet:

import uproot

tree = uproot.open({"DAOD_PHYSLITE.34857549._000351.pool.root.1": "CollectionTree"})
tree["AnalysisElectronsAuxDyn.pt"].array()

works just fine to access this specific array. A similar version with Dask fails (before a .compute()):

tree = uproot.dask({"DAOD_PHYSLITE.34857549._000351.pool.root.1": "CollectionTree"})
tree["AnalysisElectronsAuxDyn.pt"]

because parts of the file are not understandable to uproot:

UnknownInterpretation: none of the rules matched
in file DAOD_PHYSLITE.34857549._000351.pool.root.1
in object /CollectionTree;1:xTrigDecisionAux./xTrigDecisionAux.xAOD::AuxInfoBase

@lgray pointed out that this is expected behavior and coffea removes branches to address this (_remove_not_interpretable).

What I would like to raise for discussion here is making the uproot.dask behavior match more closely that of uproot.open. As long as I only need data that uproot can read, the Dask interface should be able to supply it without too much additional effort for the user. Concretely that might mean for example:

• automatically apply something like _remove_not_interpretable, possibly with an accompanying warning, or
• provide a new keyword argument and accompany the UnknownInterpretation error with information of how to use it to achieve _remove_not_interpretable-like behavior.

I do not know what the worst case scenario looks like with partially unreadable files: can this ever imply that the interpretation of the other (to uproot appearing as "readable") columns becomes wrong? If so, it is dangerous to automatically handle such files without the user potentially being aware of course.

[jpivarski]

This might be related to #1048, with the Daskified version encountering errors where an eager version does not encounter errors. If the eager version does not encounter errors, the Daskified should not encounter errors either—it's calling the same code, just at a later time on a Dask worker instead of the head node.

Oh!!! Maybe the Dask worker has an outdated version of Uproot? Maybe that's why you see different errors when running eagerly or lazily, because it's running different versions of Uproot in the two cases?

[jpivarski]

Asking for the Daskified mode to raise or not raise the same interpretation or deserialization errors (something that nothing to do with delaying computations) as the eager mode is not a feature request.

[alexander-held]

Oh!!! Maybe the Dask worker has an outdated version of Uproot? Maybe that's why you see different errors when running eagerly or lazily, because it's running different versions of Uproot in the two cases?

I was not sure how I ran this originally, but I just reproduced this locally on my laptop so I think it is not an issue with different uproot versions. A list of possibly relevant package versions (just did a new install):

Package            Version
------------------ ---------
awkward            2.5.2
awkward-cpp        28
dask               2024.1.0
dask-awkward       2024.1.2
fsspec             2023.12.2
numpy              1.26.3
uproot             5.2.1
zstandard          0.22.0

We now also have a publicly available file in the same format that can be used to reproduce the behavior above, sitting on EOS:

xrdcp root://eosuser.cern.ch//eos/user/f/feickert/physlite_public_testing/DAOD_PHYSLITE.34858087._000001.pool.root.1 .

[alexander-held]

It seems that the EOS link in my previous comment still needs some permissions to work, here is another way to access the file over cernbox that should work correctly:

curl -sLO https://cernbox.cern.ch/remote.php/dav/public-files/wJGWzAyirlWE6QV/DAOD_PHYSLITE.34858087._000001.pool.root.1