Linux mummer 3.23-pl5321hdbdd923_19 broke

[peterjc]

Unexpected side effect of #49870 which fixed macOS Intel and added macOS ARM.

Build 19 is working nicely for me on Intel Mac:

(x86 macOS) $ conda list mummer
# packages in environment at /Users/xxx/mambaforge/envs/pyani-plus_py312:
#
# Name                    Version                   Build  Channel
mummer                    3.23            pl5321h4a92bd6_19    bioconda
(x86 macOS) $ rm -rf tmp* ; nucmer -p tmp NC_002696.fasta NC_014100.fna ; echo "Return $?"
...
Return 0

But sadly this has broken on Linux, root cause currently unknown:

(x86 Linux) $ conda list mummer
# packages in environment at /home/pjacock/miniforge3/envs/pyani-plus_py312:
#
# Name                    Version                   Build  Channel
mummer                    3.23            pl5321hdbdd923_19    bioconda
(x86 Linux) $ rm -rf tmp* ; nucmer -p tmp NC_002696.fasta NC_014100.fna ; echo "Return $?"
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "tmp.ntref" of length 4016949
# construct suffix tree for sequence of length 4016949
# (maximum reference length is 18446744073709551612)
# (maximum query length is 18446744073709551615)
# process 40169 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /home/pjacock/miniforge3/envs/pyani-plus_py312/opt/mummer-3.23/mummer tmp.ntref 1.22
# reading input file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna" of length 4655622
# matching query-file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna"
# against subject-file "tmp.ntref"
ERROR: mummer and/or mgaps returned non-zero
Return 1

Reverting to build 18 works:

(x86 Linux) $ conda list mummer
# packages in environment at /home/pjacock/miniforge3/envs/pyani-plus_py312:
#
# Name                    Version                   Build  Channel
mummer                    3.23            pl5321hdbdd923_18    bioconda
(x86 Linux) $ rm -rf tmp* ; nucmer -p tmp NC_002696.fasta NC_014100.fna ; echo "Return $?"
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "tmp.ntref" of length 4016949
# construct suffix tree for sequence of length 4016949
# (maximum reference length is 2305843009213693948)
# (maximum query length is 18446744073709551615)
# process 40169 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /home/pjacock/miniforge3/envs/pyani-plus_py312/opt/mummer-3.23/mummer tmp.ntref 1.22
# reading input file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna" of length 4655622
# matching query-file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna"
# against subject-file "tmp.ntref"
# COMPLETETIME /home/pjacock/miniforge3/envs/pyani-plus_py312/opt/mummer-3.23/mummer tmp.ntref 4.69
# SPACE /home/pjacock/miniforge3/envs/pyani-plus_py312/opt/mummer-3.23/mummer tmp.ntref 8.40
4: FINISHING DATA
Return 0

Test case using build 19 fails:

$ rm -rf tmp* ; nucmer -p tmp /home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_002696.fasta /home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna ; echo "Return $?"
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "tmp.ntref" of length 4016949
# construct suffix tree for sequence of length 4016949
# (maximum reference length is 18446744073709551612)
# (maximum query length is 18446744073709551615)
# process 40169 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /home/pjacock/miniforge3/envs/pyani-plus_py312/opt/mummer-3.23/mummer tmp.ntref 1.22
# reading input file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna" of length 4655622
# matching query-file "/home/pjacock/repositories/pyani-plus/tests/fixtures/sequences/NC_014100.fna"
# against subject-file "tmp.ntref"
ERROR: mummer and/or mgaps returned non-zero
Return 1

In the short term, can we mark the Linux build 19 files as broken?

[peterjc]

Test works on macOS ARM:

(ARM macOS) $ conda list mummer
# packages in environment at /Users/peterjc/miniforge3:
#
# Name                    Version                   Build  Channel
mummer                    3.23            pl5321h85d7946_19    https://output.circle-artifacts.com/output/job/aafef683-cdb6-4af1-85da-3a41d3f21aff/artifacts/0/tmp/artifacts/packages
(ARM macOS) $ rm -rf tmp*; nucmer -p tmp NC_002696.fasta NC_010338.fna ; echo "Return $?"
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "tmp.ntref" of length 4016949
# construct suffix tree for sequence of length 4016949
# (maximum reference length is 2305843009213693948)
# (maximum query length is 18446744073709551615)
# process 40169 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /Users/peterjc/miniforge3/opt/mummer-3.23/mummer tmp.ntref 0.86
# reading input file "/Users/peterjc/repositories/pyani-plus/tests/fixtures/sequences/NC_010338.fna" of length 5477872
# matching query-file "/Users/peterjc/repositories/pyani-plus/tests/fixtures/sequences/NC_010338.fna"
# against subject-file "tmp.ntref"
# COMPLETETIME /Users/peterjc/miniforge3/opt/mummer-3.23/mummer tmp.ntref 4.30
# SPACE /Users/peterjc/miniforge3/opt/mummer-3.23/mummer tmp.ntref 9.19
4: FINISHING DATA
Return 0

I don't have immediate access to a Linux ARM machine, but would assume that broke like the x86 Linux pacakge.

[aliciaaevans]

linux-64 build 19 has been marked broken.

[peterjc]

Thank you!

Let's leave this issue open as ideally we'd be able to fix the recipe again for Linux.

[aliciaaevans]

@peterjc I played around with this a bit on linux-64, using the broken build from the PR. I can recreate that error you got, but it seems like after running the same command a few times it seems to start working. Very strange. Have you found any possible leads for why the last PR broke it?

[aliciaaevans]

Actually I take it back. Even when it appeared to work, the delta file it created was pretty much empty compared to the one from build 18.

[peterjc]

I would guess something in the compiler tool chain changed and perhaps this is the result of an undefined bit of C behaviour or something. If mummer 3 was still under active development I'd be asking the authors and/or trawling their issue tracker.

I couldn't see anything in the recipe to explain this. The new patches are restricted to osx (to make clang happy), so should have no effect on Linux. I did modify the Perl hashbangs, which includes the main mummer script and mgaps - which might be important here? The error is mummer and/or mgaps returned non-zero

[aliciaaevans]

@peterjc what was the reason for restricting the patches to osx? When I build locally on linux-64, I get those implicit declaration warnings (e.g. linkloc.c:128:5: warning: implicit declaration of function 'getbranchinfostree' [-Wimplicit-function-declaration]) which then appear to result in mummer and/or mgaps returned non-zero when the patches aren't included, but it works when the patches are included.

Those warnings are in the logs from the last Azure linux-64 build from your original PR, but my test PR doesn't have that and that package works fine on my same linux-64 while build 19 is broken. I'm wondering if something in the environment changed, but I can't find anything obvious.

In any case, if there's no harm in including the patches for Linux, I'm thinking of removing the [osx] selectors since apparently in certain circumstances they are needed.

Also the file command used to identify the perl scripts doesn't exist in the build container on Linux, but that didn't seem to be the cause of the error. Or at least, changing that didn't fix mummer and/or mgaps returned non-zero for me locally unless I also included the patches.

Let me know your thoughts, thanks.

[peterjc]

Applying the explicit declaration patches on Linux sounds fine - and if it silences warnings, great. I only applied the patch on osx because it seemed that clang treated them as failures. i.e. I was trying not to change anything on Linux as it was working.

We could explicitly make the file/sed loop which checks the hash-bangs conditional on osx?