Skip to content

Commit

Permalink
Merge pull request #2343 from easybuilders/4.3.x
Browse files Browse the repository at this point in the history
release EasyBuild v4.3.3
  • Loading branch information
boegel authored Feb 23, 2021
2 parents 92ec707 + 92cfe6c commit dd2cbce
Show file tree
Hide file tree
Showing 254 changed files with 1,477 additions and 516 deletions.
7 changes: 5 additions & 2 deletions .github/workflows/unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,18 @@ jobs:
# for modules tool
sudo apt-get install lua5.2 liblua5.2-dev lua-filesystem lua-posix tcl tcl-dev
# fix for lua-posix packaging issue, see https://bugs.launchpad.net/ubuntu/+source/lua-posix/+bug/1752082
sudo ln -s /usr/lib/x86_64-linux-gnu/lua/5.2/posix_c.so /usr/lib/x86_64-linux-gnu/lua/5.2/posix.so
# needed for Ubuntu 18.04, but not for Ubuntu 20.04, so skipping symlinking if posix.so already exists
if [ ! -e /usr/lib/x86_64-linux-gnu/lua/5.2/posix.so ] ; then
sudo ln -s /usr/lib/x86_64-linux-gnu/lua/5.2/posix_c.so /usr/lib/x86_64-linux-gnu/lua/5.2/posix.so
fi
# for testing OpenMPI-system*eb we need to have Open MPI installed
sudo apt-get install libopenmpi-dev openmpi-bin
- name: install EasyBuild framework
run: |
# first determine which branch of easybuild-framework repo to install
BRANCH=develop
if [ "x$GITHUB_BASE_REF" = 'xmaster' ]; then BRANCH=master; fi
if [ "x$GITHUB_BASE_REF" = 'xmain' ]; then BRANCH=main; fi
if [ "x$GITHUB_BASE_REF" = 'x4.x' ]; then BRANCH=4.x; fi
echo "Using easybuild-framework branch $BRANCH (\$GITHUB_BASE_REF $GITHUB_BASE_REF)"
git clone -b $BRANCH --depth 10 --single-branch https://github.com/easybuilders/easybuild-framework.git
Expand Down
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ addons:
install:
# install easybuild-framework (and dependencies)
# prefer clone & easy_install over easy_install directly in order to obtain information of what was installed exactly
# use framework 'develop' branch, except when testing 'master'
# use framework 'develop' branch, except when testing 'main'
- BRANCH=develop
- if [ "x$TRAVIS_BRANCH" = 'xmaster' ]; then BRANCH=master; fi
- if [ "x$TRAVIS_BRANCH" = 'xmain' ]; then BRANCH=main; fi
- if [ "x$TRAVIS_BRANCH" = 'x4.x' ]; then BRANCH=4.x; fi
- echo "Using easybuild-framework branch $BRANCH (\$TRAVIS_BRANCH $TRAVIS_BRANCH)"
- git clone -b $BRANCH --depth 10 --single-branch https://github.com/easybuilders/easybuild-framework.git
Expand Down
52 changes: 51 additions & 1 deletion RELEASE_NOTES
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,57 @@ For more detailed information, please see the git log.

These release notes can also be consulted at http://easybuild.readthedocs.org/en/latest/Release_notes.html.

The latest version of easybuild-easyblocks provides 227 software-specific easyblocks and 37 generic easyblocks.
The latest version of easybuild-easyblocks provides 229 software-specific easyblocks and 37 generic easyblocks.


v4.3.3 (February 23rd 2021)
---------------------------

update/bugfix release

- 2 new software-specific easyblocks:
- AOCC (#2295), Intel compilers (v2021.x, oneAPI) (#2305)
- minor enhancements, including:
- run motorBike tutorial case as sanity check for recent (community) OpenFOAM versions (#2201)
- add foamMonitor to sanity checks of OpenFOAM (#2256)
- create versioned symlinks for CMake commands + create symlink for 'cmake3' in PyTorch easyblock if cmake3 command is not found (#2259)
- add support for skipping steps in Python packages installed as extension + print progress on individual steps for installing Python packages as extensions (#2290)
- update BerkeleyGW easyblock to support GCC 10 and fftlib (#2297)
- update QuantumESPRESSO easyblock to support GCC 10 (#2298)
- update Clang easyblock to add support for building extra tools + leveraging hwloc and Z3 as optional dependencies (#2310)
- add support for running TensorFlow CPU and GPU tests (#2263, #2292, #2312)
- update impi easyblock for impi 2021.x (oneAPI) (#2313)
- update QuantumESPRESSO easyblock to handle v6.7 (#2319)
- update OpenFOAM easyblock for changes in v2012 (#2321)
- add sanity check commands to GCC (including LTO support) (#2322)
- update FLUENT sanity check for v2021R1 (#2334)
- various bug fixes, including:
- filter out user packages in LAMMPS easyblock if corresponding dependency isn't included + only set -DUSER-INTEL on x86_64 systems (#2254)
- unify handling of pylibdirs and don't add duplicated $PYTHONPATH in PythonBundle (#2281)
- enhance Amber easyblock to fix running of update_amber script when 'python' command is not available in OS (#2282)
- improve Bazel easyblock: add support for running tests, enable static linking, use build dir rather than tmpdir, verbose output (#2285)
- guard 'module unload' statements in modules for Cray* toolchains (#2286)
- set $PYTHONNOUSERSITE in PythonBundle.extensions_step to avoid picking up on Python packages installed in $HOME (#2289)
- create less temporary directories for TensorFlow by (only) using --output_user_root (#2293)
- fix logic w.r.t. enabling Python support in PETSc (#2299)
- make builddeps a list of names in SLEPc easyblock (#2300)
- make builddeps a list of names in Trilinos easyblock (#2301)
- make sure the installation of libiberty.a in the binutils easyblock goes into a populated directory (#2308)
- fix for building GCC with --sysroot on ppc64le (#2315)
- fix OpenFOAM sanity check on POWER (#2320)
- use library search paths of compiler for RPATH when building binutils with system compiler + enhance sanity check by running --version for binutils commands (#2323, #2327)
- pass $CXXFLAGS to PDT's configure script via '-useropt' (#2324)
- pass down compilation flags from build environment for ESMF (#2325)
- update URLs for test data for WRF to https (#2335)
- read MATLAB configuration file in binary mode to avoid UTF-8 encoding errors when using Python 3.6 (#2340)
- fix Boost sanity check on POWER (#2291) and aarch64 (#2341)
- other changes, including:
- rework module-only tests to use unique software name (rather than 'foo') (#2287)
- prefer default value for extra options in easyblock tests (#2280, #2302)
- add check for accepted EULA in custom easyblock for NVHPC (#2311)
- update optional feature support of TensorFlow (#2314)
- make symlinking of posix_c.so to posix.so in test suite configuration conditional (#2330)


v4.3.2 (December 10th 2020)
---------------------------
Expand Down
2 changes: 1 addition & 1 deletion easybuild/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
4 changes: 2 additions & 2 deletions easybuild/easyblocks/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down Expand Up @@ -43,7 +43,7 @@
# recent setuptools versions will *TRANSFORM* something like 'X.Y.Zdev' into 'X.Y.Z.dev0', with a warning like
# UserWarning: Normalizing '2.4.0dev' to '2.4.0.dev0'
# This causes problems further up the dependency chain...
VERSION = LooseVersion('4.3.2')
VERSION = LooseVersion('4.3.3')
UNKNOWN = 'UNKNOWN'


Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/abaqus.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/acml.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/adf.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2016-2020 Ghent University
# Copyright 2016-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/advisor.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/aladin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/allinea.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2013-2020 Ghent University
# Copyright 2013-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
21 changes: 17 additions & 4 deletions easybuild/easyblocks/a/amber.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2015-2020 Stanford University
# Copyright 2009-2021 Ghent University
# Copyright 2015-2021 Stanford University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down Expand Up @@ -38,6 +38,7 @@
from easybuild.easyblocks.generic.pythonpackage import det_pylibdir
from easybuild.framework.easyconfig import CUSTOM
from easybuild.tools.build_log import EasyBuildError
from easybuild.tools.filetools import which
from easybuild.tools.modules import get_software_root
from easybuild.tools.run import run_cmd

Expand Down Expand Up @@ -82,8 +83,20 @@ def extract_step(self):
def patch_step(self, *args, **kwargs):
"""Patch Amber using 'update_amber' tool, prior to applying listed patch files (if any)."""

# figure out which Python command to use to run the update_amber script;
# by default it uses 'python', but this may not be available (on CentOS 8 for example);
# note that the dependencies are not loaded yet at this point, so we're at the mercy of the OS here...
python_cmd = None
for cand_python_cmd in ['python', 'python3', 'python2']:
if which(cand_python_cmd):
python_cmd = cand_python_cmd
break

if python_cmd is None:
raise EasyBuildError("No suitable Python command found to run update_amber script!")

if self.cfg['patchlevels'] == "latest":
cmd = "./update_amber --update"
cmd = "%s ./update_amber --update" % python_cmd
# Run as many times as specified. It is the responsibility
# of the easyconfig author to get this right, especially if
# he or she selects "latest". (Note: "latest" is not
Expand All @@ -94,7 +107,7 @@ def patch_step(self, *args, **kwargs):
for (tree, patch_level) in zip(['AmberTools', 'Amber'], self.cfg['patchlevels']):
if patch_level == 0:
continue
cmd = "./update_amber --update-to %s/%s" % (tree, patch_level)
cmd = "%s ./update_amber --update-to %s/%s" % (python_cmd, tree, patch_level)
# Run as many times as specified. It is the responsibility
# of the easyconfig author to get this right.
for _ in range(self.cfg['patchruns']):
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/anaconda.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/ansys.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/ant.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
128 changes: 128 additions & 0 deletions easybuild/easyblocks/a/aocc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
##
# Copyright 2020-2021 Forschungszentrum Juelich GmbH
#
# This file is triple-licensed under GPLv2 (see below), MIT, and
# BSD three-clause licenses.
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
# with support of Ghent University (http://ugent.be/hpc),
# the Flemish Supercomputer Centre (VSC) (https://www.vscentrum.be),
# Flemish Research Foundation (FWO) (http://www.fwo.be/en)
# and the Department of Economy, Science and Innovation (EWI) (http://www.ewi-vlaanderen.be/en).
#
# https://github.com/easybuilders/easybuild
#
# EasyBuild is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation v2.
#
# EasyBuild is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with EasyBuild. If not, see <http://www.gnu.org/licenses/>.
##
"""
Support for installing AOCC, implemented as an easyblock.
@author: Sebastian Achilles (Forschungszentrum Juelich GmbH)
"""

import os

from easybuild.easyblocks.generic.packedbinary import PackedBinary
from easybuild.framework.easyconfig import CUSTOM
from easybuild.tools.build_log import EasyBuildError
from easybuild.tools.systemtools import get_shared_lib_ext


class EB_AOCC(PackedBinary):
"""
Support for installing the AOCC compilers
"""

@staticmethod
def extra_options():
extra_vars = {
'clangversion': [None, "Clang Version on which AOCC is based on (10.0.0 or 11.0.0 or ...)", CUSTOM],
}
return PackedBinary.extra_options(extra_vars)

def __init__(self, *args, **kwargs):
"""Easyblock constructor, define custom class variables specific to AOCC."""
super(EB_AOCC, self).__init__(*args, **kwargs)

self.clangversion = self.cfg['clangversion']

def _aocc_guess_clang_version(self):
map_aocc_to_clang_ver = {
'2.3.0': '11.0.0',
}

if self.version in map_aocc_to_clang_ver:
return map_aocc_to_clang_ver[self.version]
else:
error_lines = [
"AOCC is based on Clang. Guessing Clang version in easyblock failed.",
"You should either:",
"- specify `clangversion` in the easyconfig;",
"- extend `map_aocc_to_clang_ver` in the easyblock;",
]
raise EasyBuildError('\n'.join(error_lines))

def install_step(self):
# EULA for AOCC must be accepted via --accept-eula EasyBuild configuration option,
# or via 'accept_eula = True' in easyconfig file
self.check_accepted_eula(more_info='http://developer.amd.com/wordpress/media/files/AOCC_EULA.pdf')

# AOCC is based on Clang. Try to guess the clangversion from the AOCC version
# if clangversion is not specified in the easyconfig
if self.clangversion is None:
self.clangversion = self._aocc_guess_clang_version()

super(EB_AOCC, self).install_step()

def sanity_check_step(self):
"""Custom sanity check for AOCC, based on sanity check for Clang."""
shlib_ext = get_shared_lib_ext()
custom_paths = {
'files': [
'bin/clang', 'bin/clang++', 'bin/flang', 'bin/lld', 'bin/llvm-ar', 'bin/llvm-as', 'bin/llvm-config',
'bin/llvm-link', 'bin/llvm-nm', 'bin/llvm-symbolizer', 'bin/opt', 'bin/scan-build', 'bin/scan-view',
'include/clang-c/Index.h', 'include/llvm-c/Core.h', 'lib/clang/%s/include/omp.h' % self.clangversion,
'lib/clang/%s/include/stddef.h' % self.clangversion, 'lib/libc++.%s' % shlib_ext,
'lib/libc++abi.%s' % shlib_ext, 'lib/libclang.%s' % shlib_ext, 'lib/libomp.%s' % shlib_ext,
],
'dirs': ['include/llvm', 'lib/clang/%s/lib' % self.clangversion, 'lib32'],
}

custom_commands = [
"clang --help",
"clang++ --help",
"flang --help",
"llvm-config --cxxflags",
]
super(EB_AOCC, self).sanity_check_step(custom_paths=custom_paths, custom_commands=custom_commands)

def make_module_extra(self):
"""Custom variables for AOCC module."""
txt = super(EB_AOCC, self).make_module_extra()
# we set the symbolizer path so that asan/tsan give meanfull output by default
asan_symbolizer_path = os.path.join(self.installdir, 'bin', 'llvm-symbolizer')
txt += self.module_generator.set_environment('ASAN_SYMBOLIZER_PATH', asan_symbolizer_path)
# setting the AOCChome path
txt += self.module_generator.set_environment('AOCChome', self.installdir)
return txt

def make_module_req_guess(self):
"""
A dictionary of possible directories to look for.
Include C_INCLUDE_PATH and CPLUS_INCLUDE_PATH as an addition to default ones
"""
guesses = super(EB_AOCC, self).make_module_req_guess()
guesses['C_INCLUDE_PATH'] = ['include']
guesses['CPLUS_INCLUDE_PATH'] = ['include']
return guesses
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/arb.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/armadillo.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/a/atlas.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 Ghent University
# Copyright 2009-2021 Ghent University
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
2 changes: 1 addition & 1 deletion easybuild/easyblocks/b/bamtools.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
##
# Copyright 2009-2020 The Cyprus Institute
# Copyright 2009-2021 The Cyprus Institute
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
Expand Down
Loading

0 comments on commit dd2cbce

Please sign in to comment.