oll/han: deadlock in MPI_Intercomm_merge / MPI_Comm_split at multi-node scale

[yinliaws]

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

Open MPI main (git, snapshot around commit fa564cd6).

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Built from a git clone of main, configured with:

./configure --enable-debug --enable-mpi1-compatibility --with-ofi=<libfabric> --without-verbs

Please describe the system on which you are running

Using AWS cluster:

• 4 nodes, 8 MPI ranks per node (32 ranks total), c5n.18xlarge.
• Network: OFI / libfabric (EFA provider).
• OS: Amazon Linux 2 (Linux, x86_64).
• han is selected automatically (it is the default collective component at
  multi-node scale — it is not explicitly requested on the command line).

Details of the problem

Please describe, in detail, the problem that you are having, including the behavior you expect to see, the actual behavior that you are seeing, steps to reproduce the problem, etc. It is most helpful if you can attach a small program that a developer can use to reproduce your problem.

With the han collective component enabled (the default here), a program that performs MPI_Intercomm_create / MPI_Intercomm_merge together with MPI_Comm_split in a loop deadlocks at multi-node scale.

Reproducer: the Intel test suite from open-mpi/ompi-tests (intel_tests/src/MPI_Intercomm_merge2_c), built against this Open MPI. The Intel test-suite tests MPI_Intercomm_merge2 and MPI_Intercomm_merge3 reproduce this deterministically at 32 ranks over 4 nodes (8 ranks/node):

hangs (~100% of runs; no forward progress; must be killed):
mpirun -np 32 -N 8 ./intel_tests/src/MPI_Intercomm_merge2_c

# completes in ~2 s:
mpirun -np 32 -N 8 --mca coll ^han ./intel_tests/src/MPI_Intercomm_merge2_c

MPI_Intercomm_merge3_c behaves identically. Both hang with han and pass with --mca coll ^han, on every run observed. The hang is in the collective layer (backtraces are entirely in coll/han + coll/base), not in the BTL/MTL, so it does not appear to be transport-specific.

The triggering pattern

Per iteration, over a range of communicator configurations, the test:

1. obtains/creates a communicator (for the "split" configurations this is MPI_Comm_split(MPI_COMM_WORLD, …) — a collective over MPI_COMM_WORLD);
2. splits it again with one rank excluded via color = MPI_UNDEFINED (that rank receives MPI_COMM_NULL);
3. has the non-excluded ranks build an intercommunicator with MPI_Intercomm_create (bridge communicator = the communicator from step 1) and then MPI_Intercomm_merge it;
4. the excluded rank skips step 3 and proceeds to the next iteration.

This is standard-conforming: non-barrier collectives are not required to synchronize, and collectives on distinct communicators are independent, so the excluded rank is allowed to run ahead of the others.

Observed state at the hang

Attaching gdb to the ranks (the state is stable across repeated samples over several minutes — i.e. a genuine deadlock, not slow progress):

• Every rank is blocked in an unsatisfiable mca_pml_cm_recv inside a coll/han collective; no rank makes progress.
• On a given node (the 8 ranks forming that node's MPI_COMM_WORLD sub-group):
  • about half are in MPI_Comm_split(MPI_COMM_WORLD), blocked in a coll/han
    intra-node gather, each waiting to receive from a specific peer on the node;
  • the other half are in MPI_Intercomm_merge, blocked in coll/han
    collectives on the intercommunicator (some as the node's gather root, others
    in the following broadcast).
• The ranks blocking the Comm_split gather are the ranks tied up in
  Intercomm_merge, and vice-versa; none can reach the collective the others
  are waiting on.

Representative backtraces:

# a rank in MPI_Comm_split(MPI_COMM_WORLD):
mca_pml_cm_recv                                 (blocked recv, tag = GATHER)
ompi_coll_base_gather_intra_binomial            coll_base_gather.c
mca_coll_han_gather_intra_dynamic               coll_han_dynamic.c
mca_coll_han_allgather_intra                    coll_han_allgather.c
mca_coll_han_allgather_intra_dynamic            coll_han_dynamic.c
ompi_comm_split_with_info                       communicator/comm.c
PMPI_Comm_split

# a rank in MPI_Intercomm_merge (intra-node gather root, blocked recv):
mca_pml_cm_recv                                 (blocked recv, tag = GATHER)
ompi_coll_base_gather_intra_binomial            coll_base_gather.c
mca_coll_han_gatherv_intra                      coll_han_gatherv.c
mca_coll_han_gatherv_intra_dynamic              coll_han_dynamic.c
mca_coll_inter_allgatherv_inter                 coll/inter/coll_inter_allgatherv.c
ompi_comm_determine_first                       communicator/comm.c
PMPI_Intercomm_merge

# a rank in MPI_Intercomm_merge (broadcast phase, blocked recv):
ompi_coll_base_bcast_intra_generic              coll_base_bcast.c
mca_coll_han_bcast_intra                        coll_han_bcast.c
mca_coll_han_bcast_intra_dynamic                coll_han_dynamic.c
mca_coll_inter_allgatherv_inter                 coll/inter/coll_inter_allgatherv.c
PMPI_Intercomm_merge