Building parts of Spack environments on different nodes: option1 (include other concretised environments)

[ccaefch0523]

I created a new spacksite in Myriad using Spack 0.22 on the build01 node:

[ccspapp@build01 hpc-spack]$ spacksites/spacksites create fc-myriad-s0.22-test
/spack/0.22/fc-myriad-s0.22-test/spack/share/spack/setup-env.sh spack compiler find --scope=site
# SPACKSITES: have set SPACK_DISABLE_LOCAL_CONFIG=1
# SPACKSITES: have set HPC_SPACK_ROOT=/lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack
# SPACKSITES: Have set spacks's external compiler and python dependencies - now calling spack compiler find --scope=site
==> Added 3 new compilers to /lustre/shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/etc/spack/compilers.yaml
    gcc@4.9.2  gcc@4.8.5  gcc@11.2.1
==> Compilers are defined in the following files:
    /lustre/shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/etc/spack/compilers.yaml
# SPACKSITES: Now calling: #######
 source /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/spacksites/process-env-scripts/spack-deps-rhel-7.8.sh

source  /shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/share/spack/setup-env.sh
spack config --scope=site get config
# SPACKSITES: ####################
# SPACKSITES: site.build_stage set to:  /shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/../build_stage

gpg key trust done so I can use the buildcache

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack gpg trust /shared/ucl/apps/spack/0.22/buildcache/build_cache/_pgp/8AD9CBD92CD2A4AEB15F3458969BB097C2225210.pub
gpg: key C2225210: public key "ARCHPCSolutions (GPG created for Spack) <rc-support@ucl.ac.uk>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg: inserting ownertrust of 6

but then I had an empty list of the buildcache:

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack buildcache list --allarch
==> 0 cached builds.

so I tried updating the indexing without the flag -d and it works!

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack buildcache update-index -d /shared/ucl/apps/spack/0.22/buildcache
==> Error: unrecognized arguments: -d

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack buildcache update-index /shared/ucl/apps/spack/0.22/buildcache
(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack buildcache list --allarch
==> 134 cached builds.
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
autoconf@2.69        bzip2@1.0.8         gdbm@1.23       gmp@6.2.1        libxml2@2.10.3  pigz@2.8       xz@5.4.6
automake@1.16.5      diffutils@3.9       gettext@0.22.5  libiconv@1.17    m4@1.4.19       pkgconf@1.9.5  zlib@1.3.1
berkeley-db@18.1.40  findutils@4.9.0     glibc@2.17      libsigsegv@2.14  ncurses@6.4     readline@8.2   zstd@1.5.5
binutils@2.42        gcc-runtime@11.2.1  gmake@4.4.1     libtool@2.4.7    perl@5.38.0     tar@1.30

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
apr@1.7.4                           hdf5@1.14.3          ...

Experimenting with the myriad.yaml generated by Heather see https://github.com/UCL-ARC/hpc-spack/issues/56

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack env create -d myproject  $HPC_SPACK_ROOT/spacksites/spack-env-templates/dev1/build/myriad.yaml
==> Created independent environment in: /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/myproject
==> Activate with: spack env activate myproject

I activated my env myproject

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$  spack env activate -p myproject
[myproject] (spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack find 
==> In environment /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/myproject
==> 19 root specs
-- no arch / gcc@12.3.0 -----------------------------------------
 -  bcftools@1.16%gcc@12.3.0
 -  beast2@2.7.4%gcc@12.3.0
 -  bedtools2@2.31.0%gcc@12.3.0
 -  bwa@0.7.17%gcc@12.3.0
 -  cp2k@2023.2%gcc@12.3.0 +cuda+mpi+openmp~sirius build_system=cmake cuda_arch=80
 -  cuda@12.2.0%gcc@12.3.0
 -  dbcsr@2.6.0%gcc@12.3.0
 -  gatk@4.4.0.0%gcc@12.3.0
 -  gzip@1.13%gcc@12.3.0
 -  hdf5@1.14.3%gcc@12.3.0 +cxx+fortran+hl~mpi
 -  hdf5@1.14.3%gcc@12.3.0 +fortran+hl+mpi
 -  htslib@1.17%gcc@12.3.0
 -  netcdf-c@4.9.2%gcc@12.3.0 ~mpi
 -  netcdf-fortran@4.6.1%gcc@12.3.0
 -  openmpi@4.1.6%gcc@12.3.0  fabrics=cma,knem,ofi,psm2,ucx,xpmem schedulers=sge
 -  picard@2.26.2%gcc@12.3.0
 -  python@3.11.6%gcc@12.3.0
 -  samtools@1.17%gcc@12.3.0
 -  vcftools@0.1.16%gcc@12.3.0

==> 0 installed packages

Adding Gromacs to the spec in my env myprojectand then concretise

spack add gromacs@2023.1%gcc@12.3.0+double ^openmpi@4.1.6%gcc@12.3.0 
==> Adding gromacs@2023.1%gcc@12.3.0+double ^openmpi@4.1.6%gcc@12.3.0 to environment /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/myproject

Gromacs is added to the root specs

spack find
==> In environment /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/myproject
==> 20 root specs
-- no arch / gcc@12.3.0 -----------------------------------------
 -  bcftools@1.16%gcc@12.3.0
 -  beast2@2.7.4%gcc@12.3.0
 -  bedtools2@2.31.0%gcc@12.3.0
 -  bwa@0.7.17%gcc@12.3.0
 -  cp2k@2023.2%gcc@12.3.0 +cuda+mpi+openmp~sirius build_system=cmake cuda_arch=80
 -  cuda@12.2.0%gcc@12.3.0
 -  dbcsr@2.6.0%gcc@12.3.0
 -  gatk@4.4.0.0%gcc@12.3.0
 -  gromacs@2023.1%gcc@12.3.0 +double
 -  gzip@1.13%gcc@12.3.0
 -  hdf5@1.14.3%gcc@12.3.0 +cxx+fortran+hl~mpi
 -  hdf5@1.14.3%gcc@12.3.0 +fortran+hl+mpi
 -  htslib@1.17%gcc@12.3.0
 -  netcdf-c@4.9.2%gcc@12.3.0 ~mpi
 -  netcdf-fortran@4.6.1%gcc@12.3.0
 -  openmpi@4.1.6%gcc@12.3.0  fabrics=cma,knem,ofi,psm2,ucx,xpmem schedulers=sge
 -  picard@2.26.2%gcc@12.3.0
 -  python@3.11.6%gcc@12.3.0
 -  samtools@1.17%gcc@12.3.0
 -  vcftools@0.1.16%gcc@12.3.0

==> 0 installed packages

I need to define a rule to exclude installing/concretising cp2k in my env ???

[heatherkellyucl]

You don't need to have a rule excluding cp2k - instead it ought to be in the gpu environment.

Don't use my myriad.yaml directly, instead you need to modify it to use include_concrete instead of

  include:
  - $HPC_SPACK_ROOT/spacksites/spack-env-templates/dev1/build/base.yaml
  - $HPC_SPACK_ROOT/spacksites/spack-env-templates/dev1/build/gpu.yaml
  - $HPC_SPACK_ROOT/spacksites/spack-env-templates/dev1/build/gpu-on-gpu.yaml

so that you make separate concretised environments for base, gpu-on-gpu, gpu and then include those in myriad.yaml.

[heatherkellyucl]

(There's a question about which of the environments should include_concrete which others - so it ends up more like the chaining than what I did).

[heatherkellyucl]

For example, you concretise and build the base environment, then the gpu-on-gpu environment separately (and on a gpu node), then the gpu environment that includes the gpu-on-gpu environment, and finally the myriad environment that includes everything and which has anything extra that is only for myriad.

That's if we keep the split of how the environments are divided up the same as I initially did - if another split makes more sense, do that instead.

CPU Gromacs should go in base if we're keeping the split the same since CPU Gromacs gets installed everywhere.

[heatherkellyucl]

I got the most recent gromacs from spack develop: #44 (comment)

so we can have

gromacs@2024.2 +double
gromacs@2024.2 +cuda cuda_arch=80 
gromacs@2023 +double +plumed
gromacs@2023 +cuda cuda_arch=80 +plumed

[ccaefch0523]

Testing --include-concrete

I created a base environment added perl and python concretised and then created another environment testincluded that includes base

spack env create --include-concrete base testincluded

Spack env list
==> 3 environments
    base  base_env1  testincluded

Deleted base_env1

 spack env remove base_env1
==> Really remove environment base_env1? [y/N] y
==> Successfully removed environment 'base_env1'

spack env list
==> 2 environments
    base  testincluded

Added nedit and zlib to env base but did not concretise nedit it

spack -e base find -c

==> In environment base
==> 4 root specs
 -  nedit  [+] perl  [+] python  [+] zlib

==> Concretized roots
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
perl@5.38.0  zlib@1.3.1

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
python@3.11.6

==> Installed packages
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
berkeley-db@18.1.40  gcc-runtime@11.2.1  glibc@2.17     libxml2@2.10.3  pigz@2.8       tar@1.30
bzip2@1.0.8          gdbm@1.23           gmake@4.4.1    ncurses@6.4     pkgconf@1.9.5  xz@5.4.6
diffutils@3.9        gettext@0.22.5      libiconv@1.17  perl@5.38.0     readline@8.2   zlib@1.3.1

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
ca-certificates-mozilla@2023-05-30  libbsd@0.12.1  libxcrypt@4.4.35  sqlite@3.43.2
expat@2.6.2                         libffi@3.4.6   openssl@3.3.0     util-linux-uuid@2.38.1
gcc-runtime@12.3.0                  libmd@1.0.4    python@3.11.6
==> 29 installed packages

After concretising the base, I still don't see zlib and nedit in testincluded env . So the change is not reflected in the included environment testincluded

spack -e testincluded find -c
==> In environment testincluded
==> No root specs

==> Included specs
perl  python

==> Concretized roots

==> Installed packages
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
berkeley-db@18.1.40  gcc-runtime@11.2.1  glibc@2.17     libxml2@2.10.3  pigz@2.8       tar@1.30
bzip2@1.0.8          gdbm@1.23           gmake@4.4.1    ncurses@6.4     pkgconf@1.9.5  xz@5.4.6
diffutils@3.9        gettext@0.22.5      libiconv@1.17  perl@5.38.0     readline@8.2   zlib@1.3.1

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
ca-certificates-mozilla@2023-05-30  libbsd@0.12.1  libxcrypt@4.4.35  sqlite@3.43.2
expat@2.6.2                         libffi@3.4.6   openssl@3.3.0     util-linux-uuid@2.38.1
gcc-runtime@12.3.0                  libmd@1.0.4    python@3.11.6
==> 29 installed packages

I need to reconcretize the env testincluded

 spack -e testincluded  concretize
(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 hpc-spack]$ spack -e testincluded  find -c
==> In environment testincluded
==> No root specs

==> Included specs
nedit  perl  python  zlib

==> Concretized roots

==> Installed packages
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
autoconf@2.69         flex@2.6.3          kbproto@1.0.7         libxau@1.0.11      perl@5.38.0         xproto@7.0.31
automake@1.16.5       fontsproto@2.1.3    libfontenc@1.1.8      libxcb@1.16        pigz@2.8            xtrans@1.5.0
bdftopcf@1.1          gcc-runtime@11.2.1  libice@1.1.1          libxdmcp@1.1.4     pkgconf@1.9.5       xz@5.4.6
berkeley-db@18.1.40   gdbm@1.23           libiconv@1.17         libxfont@1.5.4     readline@8.2        zlib@1.3.1
bzip2@1.0.8           gettext@0.22.5      libpthread-stubs@0.5  libxml2@2.10.3     renderproto@0.11.1
compositeproto@0.4.2  glibc@2.17          libsigsegv@2.14       m4@1.4.19          tar@1.30
diffutils@3.9         gmake@4.4.1         libsm@1.2.4           mkfontdir@1.0.7    xbitmaps@1.1.3
findutils@4.9.0       gperf@3.1           libtool@2.4.7         mkfontscale@1.2.3  xcb-proto@1.16.0
fixesproto@5.0        inputproto@2.3.2    libx11@1.8.7          ncurses@6.4        xextproto@7.3.0

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
bison@3.8.2                         gcc-runtime@12.3.0   libxcrypt@4.4.35  sqlite@3.43.2
ca-certificates-mozilla@2023-05-30  libbsd@0.12.1        nasm@2.15.05      util-linux-uuid@2.38.1
cmake@3.27.9                        libffi@3.4.6         nghttp2@1.57.0    util-macros@1.19.3
curl@8.7.1                          libjpeg-turbo@3.0.0  openssl@3.3.0
expat@2.6.2                         libmd@1.0.4          python@3.11.6
freetype@2.13.2                     libpng@1.6.39        python-venv@1.0
==> 70 installed packages

I need to test removing a spec from base

Testing creating an independent environment via an environment files ../spack.yam placed in a directory.

 spack env create -d new-code ../spack.yaml
==> Updating view at /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/fctestenv/new-code/.spack-env/view
==> Created independent environment in: /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/fctestenv/new-code
==> Activate with: spack env activate new-code
(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 fctestenv]$ spack env list
==> 3 environments
    base  base_env1  testincluded

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 fctestenv]$ ls
new-code

the new env in the directory new-code exists

(spacksite: fc-myriad-s0.22-test) [ccspapp@build01 fctestenv]$ spack -e new-code find
==> In environment /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/fctestenv/new-code
==> No root specs

==> Included specs
perl  python

==> Installed packages
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
berkeley-db@18.1.40  gcc-runtime@11.2.1  glibc@2.17     libxml2@2.10.3  pigz@2.8       tar@1.30
bzip2@1.0.8          gdbm@1.23           gmake@4.4.1    ncurses@6.4     pkgconf@1.9.5  xz@5.4.6
diffutils@3.9        gettext@0.22.5      libiconv@1.17  perl@5.38.0     readline@8.2   zlib@1.3.1

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
ca-certificates-mozilla@2023-05-30  libbsd@0.12.1  libxcrypt@4.4.35  sqlite@3.43.2
expat@2.6.2                         libffi@3.4.6   openssl@3.3.0     util-linux-uuid@2.38.1
gcc-runtime@12.3.0                  libmd@1.0.4    python@3.11.6
==> 29 installed packages

[heatherkellyucl]

Our existing specs are split into four parts (myriad, base, gpu, gpu-on-gpu). The process for doing this and yaml files for these four parts need to be created, and the yaml files added to the repo.

Q: which environments should include which of the others to completely create this structure?

[heatherkellyucl]

Once this is working, then we can add the final packages to whichever of those files is appropriate.

[ccaefch0523]

Creating an env for Gromacs/plumed/ GPU (versions are listed below) that included base (needs editing)
spacksites/spack-env-templates/dev1/build/myriad.yaml

gromacs@2024.2 +cuda cuda_arch=80 +plumed
gromacs@2023 +cuda cuda_arch=80 +plumed

 spack env create --include-concrete base  gromacsgpu
==> Updating view at /lustre/shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/var/spack/environments/gromacsgpu/.spack-env/view

==> Created environment gromacsgpu in: /lustre/shared/ucl/apps/spack/0.22/fc-myriad-s0.22-test/spack/var/spack/environments/gromacsgpu
==> Activate with: spack env activate gromacsgpu

adding Gromacs

spack -e gromacsgpu add gromacs@2023 +cuda+plumed cuda_arch=80

spack -e gromacsgpu find -c
==> In environment gromacsgpu
==> 1 root specs
 -  gromacs@2023 +cuda+plumed cuda_arch=80

==> Included specs
-- no arch / gcc@12.3.0 -----------------------------------------
beast2@2.7.4%gcc@12.3.0      gzip@1.13%gcc@12.3.0                        netcdf-c@4.9.2%gcc@12.3.0 ~mpi                                     python@3.11.6%gcc@12.3.0
bedtools2@2.31.0%gcc@12.3.0  hdf5@1.14.3%gcc@12.3.0 +cxx+fortran+hl~mpi  netcdf-fortran@4.6.1%gcc@12.3.0                                    samtools@1.17%gcc@12.3.0
bwa@0.7.17%gcc@12.3.0        hdf5@1.14.3%gcc@12.3.0 +fortran+hl+mpi      openmpi@4.1.6%gcc@12.3.0  fabrics=cma,ofi,psm2,ucx schedulers=sge  vcftools@0.1.16%gcc@12.3.0
gatk@4.4.0.0%gcc@12.3.0      htslib@1.17%gcc@12.3.0                      picard@2.26.2%gcc@12.3.0

==> Concretized roots
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
gromacs@2023

==> Installed packages
-- linux-rhel7-cascadelake / gcc@11.2.1 -------------------------
autoconf@2.69    berkeley-db@18.1.40  diffutils@3.9    gcc-runtime@11.2.1  gettext@0.22.5  gmake@4.4.1    libsigsegv@2.14  libxml2@2.10.3  ncurses@6.4  pigz@2.8       readline@8.2  xz@5.4.6
automake@1.16.5  bzip2@1.0.8          findutils@4.9.0  gdbm@1.23           glibc@2.17      libiconv@1.17  libtool@2.4.7    m4@1.4.19       perl@5.38.0  pkgconf@1.9.5  tar@1.30      zlib@1.3.1

-- linux-rhel7-cascadelake / gcc@12.3.0 -------------------------
bison@3.8.2                         cmake@3.27.9  expat@2.6.2         libbsd@0.12.1  libmd@1.0.4       nghttp2@1.57.0  python@3.11.6    sqlite@3.43.2           util-macros@1.19.3
ca-certificates-mozilla@2023-05-30  curl@8.7.1    gcc-runtime@12.3.0  libffi@3.4.6   libxcrypt@4.4.35  openssl@3.3.0   python-venv@1.0  util-linux-uuid@2.38.1
==> 41 installed packages

[heatherkellyucl]

I have altered these yaml files to be the include_concrete versions:

	modified:   spacksites/spack-env-templates/dev1/build/base.yaml
	modified:   spacksites/spack-env-templates/dev1/build/gpu-on-gpu.yaml
	modified:   spacksites/spack-env-templates/dev1/build/gpu.yaml
	modified:   spacksites/spack-env-templates/dev1/build/myriad.yaml

Right now I have myriad.yaml including all of the other envs, but gpu including base and gpu-on-gpu including gpu. Need to test how that works.

I have also added the casteps and cpu gromacses to base.yaml and the gpu gromacses to gpu.yaml. (Don't know if that needs to be gpu-on-gpu.yaml).

[heatherkellyucl]

Starting with (in existing hk-initial-stack site):

spack env create base /home/ccspapp/Scratch/spack/0.22/hpc-spack/spacksites/spack-env-templates/dev1/build/base.yaml
==> Created environment base in: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/base
==> Activate with: spack env activate base

spack -e base concretize -f
...

spack -e base install

castep and gromacs aren't in the build cache yet so there are some things to build, starting with gsl.

==> Installing gsl-2.7.1-efih7w3hrez4pzjrvad5m3qsxks5suk4 [15/98]
==> No binary for gsl-2.7.1-efih7w3hrez4pzjrvad5m3qsxks5suk4 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/dc/dcb0fbd43048832b757ff9942691a8dd70026d5da0ff85601e52687f6deeb34b.tar.gz
==> No patches needed for gsl
==> gsl: Executing phase: 'autoreconf'
==> gsl: Executing phase: 'configure'
==> gsl: Executing phase: 'build'
==> gsl: Executing phase: 'install'
==> gsl: Successfully installed gsl-2.7.1-efih7w3hrez4pzjrvad5m3qsxks5suk4
  Stage: 3.84s.  Autoreconf: 0.00s.  Configure: 23.99s.  Build: 1m 47.60s.  Install: 4.36s.  Post-install: 1.65s.  Total: 2m 22.14s

[heatherkellyucl]

GROMACS 2024.2 on its own and Plumed 2.9.0 were fine.

==> Installing gromacs-2024.2-twuqinqf5khu2f33546sglpff5d5n6lj [93/98]
==> No binary for gromacs-2024.2-twuqinqf5khu2f33546sglpff5d5n6lj found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/80/802a7e335f2e895770f57b159e4ec368ebb0ff2ce6daccf706c6e8025c36852b.tar.gz
==> Ran patch() for gromacs
==> gromacs: Executing phase: 'cmake'
==> gromacs: Executing phase: 'build'
==> gromacs: Executing phase: 'install'
==> gromacs: Successfully installed gromacs-2024.2-twuqinqf5khu2f33546sglpff5d5n6lj
  Stage: 13.22s.  Cmake: 1m 10.63s.  Build: 5m 8.55s.  Install: 6.87s.  Post-install: 3.51s.  Total: 6m 59.72s

==> Installing plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul [95/98]
==> No binary for plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/61/612d2387416b5f82dd8545709921440370e144fd46cef633654cf0ee43bac5f8.tar.gz
==> Ran patch() for plumed
==> plumed: Executing phase: 'autoreconf'
==> plumed: Executing phase: 'configure'
==> plumed: Executing phase: 'build'
==> plumed: Executing phase: 'install'
==> plumed: Successfully installed plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul
  Stage: 14.17s.  Autoreconf: 7.77s.  Configure: 56.21s.  Build: 17m 59.95s.  Install: 45.18s.  Post-install: 1.76s.  Total: 20m 7.35s

Need to check on the patches for GROMACS 2023 (looks minor):

==> Installing gromacs-2023-i3nsau23qwhkg3c44ehsor3izfiacdqf [97/98]
==> No binary for gromacs-2023-i3nsau23qwhkg3c44ehsor3izfiacdqf found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/ac/ac92c6da72fbbcca414fd8a8d979e56ecf17c4c1cdabed2da5cfb4e7277b7ba8.tar.gz
NOTE: shell only version, useful when plumed is cross compiled
NOTE: shell only version, useful when plumed is cross compiled
PLUMED patching tool

MD engine: gromacs-2023
PLUMED location: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul/lib/plumed
diff file: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul/lib/plumed/patches/gromacs-2023.diff
sourcing config file: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/plumed-2.9.0-qfmsmjb6zdox6u5unwdkosxgn3bzqoul/lib/plumed/patches/gromacs-2023.config
Executing plumed_before_patch function
PLUMED can be incorporated into gromacs using the standard patching procedure.
Patching must be done in the gromacs root directory  _before_ the cmake command is invoked.

On clusters you may want to patch gromacs using the static version of plumed, in this case
building gromacs can result in multiple errors. One possible solution is to configure gromacs
with these additional options:

cmake -DBUILD_SHARED_LIBS=OFF -DGMX_PREFER_STATIC_LIBS=ON  

To enable PLUMED in a gromacs simulation one should use
mdrun with an extra -plumed flag. The flag can be used to  
specify the name of the PLUMED input file, e.g.:

gmx mdrun -plumed plumed.dat

For more information on gromacs you should visit http://www.gromacs.org

Linking Plumed.h, Plumed.inc, and Plumed.cmake (shared mode)
Patching with on-the-fly diff from stored originals
patching file ./cmake/gmxVersionInfo.cmake
Hunk #1 FAILED at 257.
1 out of 1 hunk FAILED -- saving rejects to file ./cmake/gmxVersionInfo.cmake.rej
patching file ./src/gromacs/CMakeLists.txt
patching file ./src/gromacs/mdlib/expanded.cpp
patching file ./src/gromacs/mdlib/expanded.h
patching file ./src/gromacs/mdlib/sim_util.cpp
patching file ./src/gromacs/mdrun/legacymdrunoptions.cpp   
patching file ./src/gromacs/mdrun/legacymdrunoptions.h
patching file ./src/gromacs/mdrun/md.cpp
patching file ./src/gromacs/mdrun/minimize.cpp
patching file ./src/gromacs/mdrun/replicaexchange.cpp
patching file ./src/gromacs/mdrun/replicaexchange.h
patching file ./src/gromacs/mdrun/rerun.cpp
patching file ./src/gromacs/mdrun/runner.cpp
patching file ./src/gromacs/modularsimulator/expandedensembleelement.cpp
patching file ./src/gromacs/taskassignment/decidegpuusage.cpp
patching file ./src/gromacs/taskassignment/include/gromacs/taskassignment/decidegpuusage.h

PLUMED is compiled with MPI support so you can configure gromacs-2023 with MPI
==> Ran patch() for gromacs
==> gromacs: Executing phase: 'cmake'
==> gromacs: Executing phase: 'build'
==> gromacs: Executing phase: 'install'
==> gromacs: Successfully installed gromacs-2023-i3nsau23qwhkg3c44ehsor3izfiacdqf
  Stage: 12.95s.  Cmake: 49.70s.  Build: 4m 56.29s.  Install: 6.54s.  Post-install: 1.67s.  Total: 6m 8.45s

[heatherkellyucl]

Plumed 2.9.0: "Patch for GROMACS 2023 (preliminary, in particular for replica-exchange, expanded ensemble, hrex features)."
Plumed 2.9.2: "Patch for GROMACS 2023 updated to the latest version"

=> we want the 2.9.2 plumed for GROMACS 2023.x instead.

The actual failure was only the version patching in cmake as in plumed/plumed2#960 (comment) but other functionality may not be there.