AiiDA Dependency Management
This page contains guidelines, concrete procedures, and any other information relevant to the management of dependencies for the aiida-core package and plugins.
The dependencies for the aiida-core package must be managed according to AEP 002.
Dependency Manager (DM):
- As of 2020-02: @csadorf
- As of 2022-10: @sphuber
- As of 2024: @agoscinski The current dependency manager should be a member of the dependency-manager team.
The dependencies for the aiida-core package are specified in the pyproject.toml file are divided into main dependencies in dependencies list and optional dependencies (extras) in project.optional-dependencies table.
Furthermore, dependencies are also specified in
-
environment.yml: to create a conda environment -
uv.lock: lockfile that we use for development and CI
The specification of a dependency within the pyproject.toml is to be considered authoritative.
In addition to above mentioned files, the DM must also be concerned with the modification of the following files:
-
utils/dependency_management.py: Utility script used for dependency management related tasks. -
.github/workflows/ci.yml: Specifies the CI workflow that checks consistency for dependency-specifying files (pyproject.toml,uv.lock, etc.). -
.github/workflows/test-install.yml: Specifies the a workflow that tests for the ability to install aiida-core with the given dependencies within the current ecosystem.
The utils/dependency_management.py script within the aiida-core repository provides commands to validate the consistency of aforementioned files with the pyproject.toml file as well as commands to re-generate them after updating a dependency.
Before adding a new dependency for the aiida-core package, make sure that the dependency adheres to the following requirements:
The newly introduced dependency
- is needed to close a non-trivial feature gap that could not be resolved easily otherwise,
- supports all Python versions supported by aiida-core (as specified in pyproject.toml),
- is available on PyPI and conda-forge [1],
- appears to be within a stable development stage, e.g., has reached a version 1.0 for projects that follow semantic versioning.
- uses an MIT-compatible license (e.g. MIT, BSD, Apache, LGPL but not GPL)
[1] In case that a dependency is not yet available through aforementioned channels, but is considered critical enough to be included anyways, the following steps can be taken:
- For lightweight dependencies, consider to vendor the package if permissible by license.
- Create a request to the current maintainer to make the dependency available open-source/on PyPI/on conda-forge.
- Maintain the PyPI project and conda-forge recipes ourselves.
This is the standard workflow for updating the aiida-core dependencies:
Modify the affected entries in pyproject.toml and then update all dependent files with
./util/dependency_management.py generate-all
The command will also be executed by the pre-commit hook (if installed).
For packages that are named differently between PyPI and conda-forge, you might need to add an entry to the SETUPTOOLS_CONDA_MAPPINGS variable within the same script.
TBD
The consistency of the various files with pyproject.toml is checked with a pre-commit hook that is executing the generate-all and the validate-* commands of utils/dependency_management.py. This hook is also executed for all commits as part of the aiida-core CI workflow.
Any changes to pyproject.toml file also trigger the test-install workflow, that explicitly checks whether the package can be installed with pip and conda. In addition, the test-install workflow is executed nightly to ensure that changes within the Python ecosystem do not lead to issues with installation or cause tests to fail.
The set of dependencies that are currently constrained and where we should make an effort towards loosening them is maintained in the form of GitHub issues with the topic/dependenciesc/constraint label.