MPI Sessions: repeated MPI_Session_init after full finalize deadlocks on macOS (shmem backing-file ENOENT + PMIx fence hang)

[jsquyres]

Summary

Repeated MPI Sessions usage — MPI_Session_init after a prior MPI_Session_finalize has dropped the instance refcount to zero — intermittently deadlocks on macOS. The second MPI_Session_init fails to open() the sm/OPAL shmem backing file with ENOENT and then hangs forever in the modex PMIx_Fence_nb wait.

This reproduces on a clean main (no PRs applied), so it is a pre-existing bug, not tied to any in-flight work.

The stock example examples/hello_sessions_c (which does two MPI_Session_init/MPI_Session_finalize cycles) is enough to trigger it; it is what surfaced this, hanging the ompi-pr-builds macOS job.

Environment

• Open MPI main (6.1.0a1), e.g. f267e427cf; embedded OpenPMIx cb89026, internal PRRTE/hwloc/libevent.
• GitHub Actions macos-15 runner, arm64, 3 logical CPUs, macOS 15.7.7 (24G720).
• mpirun --map-by ppr:1:core ./hello_sessions_c → 3 ranks.
• Built with .opal_ignore removed (all components), but that is not required.

Reproducer

Either the stock example:

/* examples/hello_sessions_c.c: MPI_Session_init/finalize twice */

or, to hit it faster, a loop of full init/finalize cycles:

#include <mpi.h>
#include <stdlib.h>
int main(int argc, char **argv) {
    int iters = (argc > 1) ? atoi(argv[1]) : 100;
    for (int i = 0; i < iters; i++) {
        MPI_Session s;
        MPI_Session_init(MPI_INFO_NULL, MPI_ERRORS_RETURN, &s);
        MPI_Session_finalize(&s);
    }
    return 0;
}

Run repeatedly with multiple local ranks:

mpirun --map-by ppr:1:core ./hello_sessions_c        # loop this; hangs within a handful of runs

A self-contained diagnostic CI workflow that builds Open MPI on macos-15, hammers the example, and on the first hang captures sample/lldb backtraces of every stuck rank + prte and dumps the session directories is here: jsquyres#30 (based on clean main).

Symptom

All ranks print the OPAL shmem/mmap "sys call fail" help on the second MPI_Session_init:

A system call failed during shared memory initialization that should not have.
  System call: open(2)
  Error:       No such file or directory (errno 2)

then the surviving ranks deadlock.

Where it hangs

Both hung ranks are in the second MPI_Session_init (hello_sessions_c.c:17), spinning in the modex fence wait:

main  (hello_sessions_c.c:17)
  MPI_Session_init
    ompi_mpi_instance_init + 3528
      usleep(100)          <-- ~99% of samples

That usleep is OMPI_LAZY_WAIT_FOR_COMPLETION(active) (ompi/runtime/ompi_rte.h:276, while (active) { opal_progress(); usleep(100); }) guarding the modex PMIx_Fence_nb(...) in ompi/instance/instance.c (~line 614). The fence completion callback (fence_release) never fires, so active never clears. prterun itself is healthy/idle (in kevent).

Analysis

The first MPI_Session_finalize takes the instance refcount to zero and tears the entire runtime down — PMIx finalize and removal of the PRRTE/OPAL session directory. The second MPI_Session_init then runs against a torn-down session directory, so:

1. the sm BTL's OPAL shmem backing-file open() lands in a directory that no longer exists → ENOENT, and
2. the modex PMIx_Fence_nb never completes → the lazy-wait loop spins forever.

In other words, a full MPI_Session_finalize → MPI_Session_init round-trip does not correctly re-establish the session directory / PMIx fence machinery (at least on macOS). This looks like an OMPI instance ↔ PMIx/PRRTE session-directory lifecycle issue.

Notes

• Intermittent/probabilistic — it hit iteration 3 of the stock example in the linked run, but the timing window depends on load (a 3-vCPU runner makes it more likely).
• The sessions_stress loop reproducer makes it far easier to hit.

[jsquyres]

The LLM was a little over-broad on this posting. We're discussing over at openpmix/openpmix#3932

---

Root cause found — this might be an OpenPMIx bug, not a session-directory issue

Follow-up investigation shows the original analysis above (torn-down
session directory / sm backing file) is not the actual cause of the
hang. Correcting the record:

• The job/session directory is intact when the hang occurs (verified: the
  parent directory of the failing open() still exists; OMPI's session-dir
  destroy flags are all false because those dirs are PMIx-provided).
• The sm shmem ENOENT is real but non-fatal: the BML (bml_r2)
  tolerates a BTL add_procs failure and continues, so MPI_Session_init
  returns SUCCESS. It's a red herring for the deadlock.
• The actual deadlock is a collective-fence desynchronization. The
  per-instance modex PMIx_Fence in ompi_mpi_instance_init is a
  whole-namespace barrier. Under repeated MPI_Session_init /
  MPI_Session_finalize cycles with timing skew, the ranks drift by whole
  iterations; the fastest rank completes all its sessions and exits, and
  the remaining ranks hang forever in a fence the departed rank was still
  expected to join.

The sm BTL is only the trigger — it adds enough per-iteration latency
to let the drift accumulate. With sm disabled
(--mca btl self,tcp --mca pml ob1 --mca smsc none) the reproducer is
100% reliable under the same load.

It's in OpenPMIx, and reproduces with no MPI at all

Because OMPI does a full PMIx_Finalize + PMIx_Init on every session
cycle (ompi_rte_finalize → PMIx_Finalize), the trigger is the PMIx
server's handling of a client that finalizes and re-initializes while
namespace-wide fences are running. lost_connection() prunes a
gracefully-finalized (and immediately reconnecting) proc from an
in-progress collective, completing it short with
PMIX_ERR_PARTIAL_SUCCESS and desynchronizing the fence sequence.

This reproduces standalone with a ~40-line pure-PMIx client under
test/simple/simptest — no Open MPI, no sm BTL. Details, mechanism, and
a fix are here:

• OpenPMIx issue: PMIx_Fence deadlocks under repeated PMIx_Init/PMIx_Finalize: lost_connection() prunes a gracefully-finalized, reconnecting proc openpmix/openpmix#3931
• OpenPMIx PR: ptl/base: don't prune a gracefully-finalized proc from collectives openpmix/openpmix#3932

Fix path for Open MPI

The fix lands on OpenPMIx master and needs to be cherry-picked to
OpenPMIx v6.1, which is the PMIx branch that Open MPI main and
v6.0.x embed (submodule pinned at v6.1.1rc2 / cb89026) — the branches
that ship MPI Sessions. Once merged + cherry-picked, this is resolved in
Open MPI via the usual PMIx submodule bump; no OMPI/OPAL code change is
required. (v5.0.x embeds PMIx v5.0 and predates the Sessions model, so
it's not affected via this path.)

With the OpenPMIx patch applied to the embedded PMIx, the
hello_sessions_c / repeated-MPI_Session_init reproducer goes from
hanging on ~run 1 to 30/30 runs × 300 cycles (9,000 cycles/rank) clean
under heavy load.

[jsquyres]

The LLM was a little over-broad on this posting. We're discussing over at openpmix/openpmix#3932

[jsquyres]

openpmix/openpmix#3932 was not the correct fix. openpmix/openpmix#3934 is at least part of the correct fix, and was merged to PMIx master. Also working on more followups. Will update here when the issue is resolved.

[rhc54]

Issue should be fixed on head of PMIx master.

[jsquyres]

Thanks @rhc54! I validated the fix by building OMPI against openpmix master -- I used the reproducer program on this issue and bumped the count up to 10K, and it worked just fine. Sweet!

How difficult would it be to get these fixes on the openpmix v6.1 branch (for OMPI main and v6.0.x branches)?

[rhc54]

How difficult would it be to get these fixes on the openpmix v6.1 branch (for OMPI main and v6.0.x branches)?

This would be something we would need to discuss - the changes are large, which is something we generally try to avoid going into an existing release branch. On the other hand, it is a correction not a new feature 🤷‍♂️

We've had some pushback about the amount of change in PMIx release series, so definitely not a slamdunk decision.