Using Mantid 6.14 is broken in some unknown way

[jl-wynen]

When I update Mantid to 6.14 in .buildconfig/ci-linux.yml and create an environment, the Jupyter kernel tends to 'die' in some situations. This does not happen with Mantid 6.13.
I get these failures locally and in CI, e.g.,: https://github.com/scipp/scippneutron/actions/runs/19627948536

For example, in docts/tutorials/3_understanding-event-data.ipynb, we have this code:

import scipp as sc
import scippneutron as scn
import scippneutron.data
import plopp as pp

dg = scn.data.tutorial_event_data()
dg  # (1)

#%

events = dg['events']
events.hist(spectrum=500, tof=400).plot()  # (2)

#%

events

#% 

sc.show(events)

#%

x = events.values[500]
x # (3)

Line (3) makes the kernel die. But if I remove lines (1) or (2), it runs fine. I traced the failure to the retrieve function in formatting_html.py (modified with extra outputs here for debugging):

def retrieve(
    var: Variable | DataArray, has_variances: bool = False, single: bool = False
) -> Any:
    if not has_variances:
        res = var.value if single else var.values
    else:
        res = var.variance if single else var.variances

    # this triggers the error:
    # print(res)

    # if we return a new array here, the error goes away:
    # res = np.array([1, 2])

    return res  # this will be passed to `repr` which then kills the kernel

When running line (3) in the above notebook, res is a numpy array. I don't know how it is possible for repr(res) to trigger such an error. Or how that error is affected by other lines in the notebook.

Here is a diff of conda environments for Mantid 6.13 and 6.14:

123,124c123,124
<   + libboost                                   1.84.0  h6c02f8c_7               conda-forge        3MB
<   + libboost-python                            1.84.0  py311h5b7b71f_7          conda-forge      123kB
---
>   + libboost                                   1.88.0  hed09d94_6               conda-forge     Cached
>   + libboost-python                            1.88.0  py311h1d5f577_6          conda-forge     Cached
190c190
<   + mantid                                     6.13.0  py311h281d3ad_0          mantid            41MB
---
>   + mantid                                     6.14.0  py311hd1ad81e_0          mantid          Cached
213c213
<   + occt                                        7.9.1  novtk_h185fe95_102       conda-forge       25MB
---
>   + occt                                        7.9.2  novtk_h185fe95_101       conda-forge     Cached

None of these package should affect the output in the above notebook because the notebook does not use mantid.

For some extra weirdness, after a kernel has died in an interactive Jupyter session, using the git CLI program in the same terminal segfaults and fails to unlock the .git folder. This goes away if I deactivate the conda env or open a new terminal.

[jokasimr]

The last part about git is really the cherry on top 😄🍒
Good job keeping your sanity through that debugging session!

[nvaytet]

🤯

Does installing Mantid install something like gitpython which could mess with the git folder?

[jl-wynen]

I've got to update the git issue: It is enough to activate the conda env to make git crash. Mantid must modify the env somehow. Turns out, the env sets LD_PRELOAD to include libjemalloc.so.2 from conda. This is responsible for crashing git.

Does installing Mantid install something like gitpython which could mess with the git folder?

It does not.

[jl-wynen]

See the linked issue in Mantid. This is all due to setting LD_PRELOAD to load jemalloc.