Updates: Move quantum-espresso to conda, add aiida-plugins, add aiida…

… example scripts (#212

- Update build tool to VirtualBox version 7 (https://www.virtualbox.org/wiki/Changelog-7.0)
- Install quantum-espresso via Conda (rather than direct compilation)
- Add more AiiDA plugins/codes, compatible with aiida-core v2, see #211
- Add `verdi code test` run, after code creation, to test it is working correctly
- Add `aiida-examples` scripts, these run a basic calculation example (with MPI), and test it completes correctly
  - currently only qe and nwchem, then more will be added later

.pre-commit-config.yaml

@@ -18,3 +18,8 @@ repos:
  hooks:
  - id: ansible-lint
  args: [--exclude, roles/]
+
+- repo: https://github.com/psf/black
+ rev: 23.1.0
+ hooks:
+ - id: black

docs/conf.py

@@ -20,7 +20,14 @@
 
 # -- General configuration ---------------------------------------------------
 
-extensions = ["myst_parser", "sphinx_design", "ablog", "sphinx.ext.intersphinx", "sphinxext.rediraffe", "sphinx_subfigure"]
+extensions = [
+ "myst_parser",
+ "sphinx_design",
+ "ablog",
+ "sphinx.ext.intersphinx",
+ "sphinxext.rediraffe",
+ "sphinx_subfigure",
+]
 
 myst_enable_extensions = ["colon_fence", "deflist", "html_image"]
 
@@ -30,7 +37,13 @@
 post_redirect_refresh = 1
 post_auto_excerpt = 2
 fontawesome_included = True
-html_sidebars = {"releases/index": ['ablog/tagcloud.html', 'ablog/archives.html', 'sbt-sidebar-nav.html']}
+html_sidebars = {
+ "releases/index": [
+ "ablog/tagcloud.html",
+ "ablog/archives.html",
+ "sbt-sidebar-nav.html",
+ ]
+}
 
 # The master toctree document.
 master_doc = "index"
@@ -63,4 +76,4 @@
  "use_edit_page_button": True,
 }
 
-rediraffe_redirects = 'redirects.txt'
+rediraffe_redirects = "redirects.txt"

docs/maintainers/release.md

@@ -61,14 +61,18 @@ tox -e update-vagrant
 Follow the [desktop build instructions](../developers/build-vagrant.md).
 
 :::{note}
-Manual modifications required, as of QM 20.03.1:
+Manual modifications required, as of QM 23.04.01, desktop shortcuts mut be manually enabled:
 
-* Double-click on the Desktop symbols to show the icons (see [#150](https://github.com/marvel-nccr/quantum-mobile/issues/150)) or just run in the VM terminal (press F5 after to refresh):
+- First install the file with <https://manpages.ubuntu.com/manpages/focal/en/man1/desktop-file-edit.1.html>:
 
- ```bash
- for FILE in ${HOME}/Desktop/*.desktop; do gio set "$FILE" "metadata::trusted" yes; done
+ ```console
+ $ sudo desktop-file-install --delete-original ~/Desktop/homepage.desktop
+ $ sudo desktop-file-install --delete-original ~/Desktop/jupyterlab.desktop
  ```
 
+- Now navigate to the `/usr/share/applications` in the folder app, find the files and drag/drop them to the Desktop
+- Right click on the file on the Desktop and select `Allow Launching`
+
 :::
 
 ### Cloud Edition
@@ -77,7 +81,7 @@ Follow the [cloud build instructions](../developers/build-cloud.md).
 
 * For the server to build the VM, choose e.g. 2 CPUs with 4GB of RAM, with at least 16GB of storage.
  * Your configuration may also be the standard configuration shown to users creating a VM using your image
- * Currently Ubuntu Server 18.04 LTS
+ * Currently Ubuntu Server 20.04 LTS
  * Expose ports as listed in instructions
  * You won't need it for long, so price for CPU/RAM is of no concern
 * Exemplary metadata:

docs/releases/versions/23.04.03.md

@@ -0,0 +1,138 @@
+---
+blogpost: true
+author: Quantum Mobile
+date: 2023-04-03
+category: Full
+tags: desktop,docker
+---
+
+# v23.04.03
+
+A Virtual Machine for Computational Materials Science
+
+## Installation instructions
+
+Get Quantum Mobile running on your computer in three simple steps:
+
+ 1. Download virtual machine image (5.9 GB)
+ - URL: <https://drive.google.com/file/d/1dhBANfO7KdaFWu_rwkBduKYNjofu8k33/view?usp=sharing>
+ - Filename: `quantum_mobile_23.04.03.ova`
+ - MD5 hash: `3d489840c77b27afe1d2367abb5ac44d`
+ 2. Install Virtual Box 6.1.6 or later (see <https://www.virtualbox.org>)
+ 3. Import virtual machine image into Virtualbox (15.7 GB)
+ File => Import Appliance
+
+Login credentials: username: `max`, password: `moritz`. 
+The default configuration of `2` cores and `1536` MB RAM can be adjusted in the VM settings.
+
+Alternatively, this Quantum Mobile is available as a Docker image: <https://hub.docker.com/r/marvelnccr/quantum-mobile>
+
+## Contact
+
+For issues encountered during installation, please first consult the [FAQ page](https://github.com/marvel-nccr/quantum-mobile/wiki/Frequently-Asked-Questions#virtualbox-installationstartup-issues).
+
+Please direct inquiries regarding Quantum Mobile to the [AiiDA mailinglist](http://www.aiida.net/mailing-list/)
+
+## Changelog
+
+- Update build tool to VirtualBox version 7 (https://www.virtualbox.org/wiki/Changelog-7.0)
+- Install quantum-espresso via Conda (rather than direct compilation)
+- Add more AiiDA plugins/codes, compatible with aiida-core v2 (see <https://github.com/marvel-nccr/quantum-mobile/issues/211>)
+- Add `verdi code test` run, after code creation, to test it is working correctly
+- Add `aiida-examples` scripts folder, these run a basic calculation example (with MPI), and test it completes correctly
+
+### Build Process
+
+- OS: `MacOSX`
+- Ansible: `2.10.17`
+- Vagrant: `2.3.4`
+- Virtualbox: `7.0.6r155176`
+- Base VM Image: `bento/ubuntu-20.04`
+
+### Software Summary
+
+```ini
+[Quantum Mobile]
+version = 23.04.03
+Operating System = Ubuntu 20.04.6 LTS
+Login credentials = max / moritz
+
+[Apt packages]
+grace = 1:5.1.25-7build1
+xcrysden = 1.6.2-3build1
+default-jre = 2:1.11-72
+rabbitmq-server = 3.8.2-0ubuntu1.4
+postgresql-client = 12+214ubuntu0.1
+
+[Conda 'aiida' environment]
+aiida-abinit = 0.4.0-pyhd8ed1ab_0@conda-forge
+aiida-core = 2.2.2-pyh1a96a4e_1@conda-forge
+aiida-core.notebook = 2.2.2-pyh1a96a4e_1@conda-forge
+aiida-cp2k = 2.0.0-pyhd8ed1ab_1@conda-forge
+aiida-nwchem = 2.1.0-pyhd8ed1ab_0@conda-forge
+aiida-pseudo = 1.0.0-pyhd8ed1ab_0@conda-forge
+aiida-quantumespresso = 4.2.0-pyhd8ed1ab_0@conda-forge
+aiida-siesta = 2.0.0-pyhd8ed1ab_0@conda-forge
+ipykernel = 6.22.0-pyh210e3f2_0@conda-forge
+jupyterlab = 3.5.3-pyhd8ed1ab_0@conda-forge
+jupyterlab-spellchecker = 0.7.3-pyhd8ed1ab_0@conda-forge
+jupyterlab-tour = 3.1.4-pyhd8ed1ab_0@conda-forge
+mamba_gator = 5.2.0-pyhd8ed1ab_0@conda-forge
+pip = 23.0.1-pyhd8ed1ab_0@conda-forge
+python = 3.9.16-h2782a2a_0_cpython@conda-forge
+
+[Conda 'abinit' environment]
+abinit = 9.8.3-hd1b6b71_2@conda-forge
+libxc = 4.3.4-h86c2bf4_2@conda-forge
+mpich = 4.0.3-h846660c_100@conda-forge
+
+[Conda 'bigdft' environment]
+bigdft-suite = 1.9.3-mpi_mpich_py311h71b1498_0@conda-forge
+libxc = 4.3.4-h86c2bf4_2@conda-forge
+mpich = 4.0.3-h846660c_100@conda-forge
+
+[Conda 'cp2k' environment]
+cp2k = 9.1.0-py39_openmpi_0@conda-forge
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'fleur' environment]
+fleur = 6.1-h005d346_1@conda-forge
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'nwchem' environment]
+libxc = 5.2.3-py39hea1df8f_2@conda-forge
+nwchem = 7.0.2-py39hea0d9f8_3@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+
+[Conda 'qespresso' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+qe = 7.0-he8a42d8_1@conda-forge
+
+[Conda 'siesta' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+siesta = 4.1.5-mpi_openmpi_hfab99a0_2@conda-forge
+
+[Conda 'yambo' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+yambo = 5.0.4-h6b7a505_1@conda-forge
+
+[Conda 'wannier90' environment]
+libxc = 5.2.3-py311h9e0c992_2@conda-forge
+openmpi = 4.1.2-hbfc84c5_0@conda-forge
+wannier90 = 3.1.0-hb97063f_2@conda-forge
+
+[Conda 'visualise' environment]
+cif2cell = 2.0.0a3-pyhd8ed1ab_0@conda-forge
+gnuplot = 5.4.5-h142138f_1@conda-forge
+jmol = 14.32.10-ha770c72_0@conda-forge
+python = 3.9.16-h2782a2a_0_cpython@conda-forge
+
+[Pseudopotentials]
+SSSP/PBE/efficiency/1.1 = /usr/local/share/pseudo_sssp_PBE_efficiency_1.1
+DOJO/PBE/FR/standard/0.4/psml = /usr/local/share/pseudo_dojo_PBE_FR_standard_0.4_psml
+```

inventory.yml

@@ -11,7 +11,7 @@ all:
  vars:
  # VM Metadata
  vm_name: "Quantum Mobile"
- vm_version: "23.03.01"
+ vm_version: "23.04.03"
  vm_description: "A Virtual Machine for Computational Materials Science"
  vm_url: "https://github.com/marvel-nccr/marvel-virtualmachine"
  vm_author: "MARVEL NCCR and MaX CoE"

local/modules/apt_show.py

@@ -54,7 +54,9 @@ def _main():
  module = AnsibleModule(argument_spec={"name": {"required": True, "type": "str"}})
  if not module.params["name"]:
  module.fail_json(msg="Package name is empty")
- _, stdout, _ = module.run_command(["apt", "show"] + module.params["name"].split(), check_rc=True)
+ _, stdout, _ = module.run_command(
+ ["apt", "show"] + module.params["name"].split(), check_rc=True
+ )
  data = {}
  lines = stdout.splitlines()
  index = 0

local/modules/verdi_code.py

@@ -34,6 +34,9 @@
  append_text:
  description: The append text
  required: false
+ run_test:
+ description: Whether to run verdi code test after creating the code (default: true)
+ required: false
 """
 
 EXAMPLES = """
@@ -66,6 +69,7 @@ def _main():
  "description": {"required": False, "type": "str"},
  "prepend_text": {"required": False, "type": "str"},
  "append_text": {"required": False, "type": "str"},
+ "run_test": {"required": False, "type": "bool", "default": True},
  },
  )
  verdi = module.params["verdi"].split() + ["--profile", module.params["profile"]]
@@ -93,6 +97,26 @@ def _main():
 
  _, stdout, _ = module.run_command(command, check_rc=True)
 
+ if "run_test" in module.params and not module.params["run_test"]:
+ # if not running test, then just exit
+ module.exit_json(changed=True, stdout=stdout)
+
+ # test the code actually works, e.g. that the executable exists
+ # otherwise delete
+ test_rc, test_stdout, test_stderr = module.run_command(
+ verdi + ["code", "test", module.params["label"]]
+ )
+ if test_rc != 0:
+ module.run_command(
+ verdi + ["code", "delete", "--force", module.params["label"]]
+ )
+ module.fail_json(
+ msg="verdi code test failed",
+ rc=test_rc,
+ stdout=test_stdout,
+ stderr=test_stderr,
+ )
+
  module.exit_json(changed=True, stdout=stdout)
 
 

local/tasks/aiida-example-run.yml

@@ -0,0 +1,14 @@
+- name: "Copy aiida script: {{ aiida_script }}"
+ ansible.builtin.copy:
+ src: aiida_run_scripts/{{ aiida_script }}
+ dest: "{{ aiida_script_path }}"
+ owner: "{{ vm_user }}"
+ group: "{{ vm_user }}"
+ register: aiida_script_copy
+
+- name: "Run aiida script: {{ aiida_script }}"
+ when: aiida_script_copy.changed
+ shell: |
+ {{ aiida_verdi_cmd }} run {{ aiida_script_path }}
+
+# TODO handle failures (delete the script?)

local/tasks/aiida-examples.yml

@@ -0,0 +1,23 @@
+- name: "Expand {{ aiida_examples_folder }}"
+ become: true
+ become_user: "{{ vm_user }}"
+ chrisjsewell.conda.expandpath:
+ path: "{{ aiida_examples_folder }}"
+ register: aiida_examples_dir
+
+- name: "Create {{ aiida_examples_dir.path }} folder"
+ ansible.builtin.file:
+ path: "{{ aiida_examples_dir.path }}"
+ state: directory
+
+# TODO get list of all files in aiida_run_scripts
+
+- name: "Copy and run aiida script"
+ include_tasks: aiida-example-run.yml
+ vars:
+ aiida_script_path: "{{ aiida_examples_dir.path }}/{{ aiida_script }}"
+ loop:
+ - qe_pw.py
+ - nwchem.py
+ loop_control:
+ loop_var: aiida_script

local/tasks/aiida-pseudo-dojo.yml

@@ -0,0 +1,53 @@
+# download pseudopotials and install them in aiida
+
+- set_fact:
+ pseudo_folder: "{{ aiida_pseudo_base_folder }}/pseudo_dojo_{{ aiida_pseudo_functional }}_{{ aiida_pseudo_relativistic }}_{{ aiida_pseudo_protocol }}_{{ aiida_pseudo_version }}_{{ aiida_pseudo_format }}"
+
+- name: create the folder to download
+ become: true
+ become_user: "{{ root_user }}"
+ file:
+ path: "{{ pseudo_folder }}"
+ state: directory
+ owner: "{{ vm_user }}"
+ group: "{{ vm_user }}"
+ mode: 0755
+
+- name: Download the archive to the download location
+ become: true
+ become_user: "{{ vm_user }}"
+ shell: |
+ {{ aiida_pseudo_cmd }} \
+ install pseudo-dojo --traceback --download-only \
+ --version {{ aiida_pseudo_version | quote }} \
+ --functional {{ aiida_pseudo_functional | quote }} \
+ --protocol {{ aiida_pseudo_protocol | quote }} \
+ --relativistic {{ aiida_pseudo_relativistic | quote }} \
+ --pseudo-format {{ aiida_pseudo_format | quote }} \
+ args:
+ chdir: "{{ pseudo_folder }}"
+ creates: "archive.tar.gz"
+
+- name: Add the archive to the AiiDA profile
+ become: true
+ become_user: "{{ vm_user }}"
+ shell: |
+ {{ aiida_pseudo_cmd }} \
+ --profile {{ aiida_pseudo_profile | quote }} \
+ install pseudo-dojo --traceback \
+ --version {{ aiida_pseudo_version | quote }} \
+ --functional {{ aiida_pseudo_functional | quote }} \
+ --protocol {{ aiida_pseudo_protocol | quote }} \
+ --relativistic {{ aiida_pseudo_relativistic | quote }} \
+ --pseudo-format {{ aiida_pseudo_format | quote }} \
+ register: result
+ failed_when: result.rc != 0 and "already installed" not in result.stderr
+ changed_when: '"already installed" not in result.stderr'
+
+- name: Document potential
+ include_role:
+ name: release_notes
+ vars:
+ section: "Pseudopotentials"
+ option: "DOJO/{{ aiida_pseudo_functional }}/{{ aiida_pseudo_relativistic }}/{{ aiida_pseudo_protocol }}/{{ aiida_pseudo_version }}/{{ aiida_pseudo_format }}"
+ value: "{{ pseudo_folder }}"