First pass for the user documentation

This adds a skeleton of the documentation, and uses sphinx-gallery
to render tutorials online.

Co-authored-by: Jigyasa Nigam <[email protected]>
Co-authored-by: HowWeiBin <[email protected]>
Co-authored-by: Philip Loche <[email protected]>

.github/workflows/tests.yml

@@ -78,7 +78,7 @@ jobs:
  - name: install tests dependencies
  run: |
  python -m pip install --upgrade pip
- python -m pip install tox black flake8
+ python -m pip install tox
 
  - name: run tests
  run: cargo test --target ${{ matrix.rust-target }} ${{ matrix.cargo-build-flags }}

docs/requirements.txt

@@ -1,10 +1,14 @@
 sphinx >=4.4
 sphinx_autodoc_typehints
-pygments >=2.9
-breathe >=4.33
-furo
+pygments >=2.9 # syntax highligthing
+breathe >=4.33 # C and C++ => sphinx through doxygen
+sphinx-gallery # convert python files into nice documentation
+furo # sphinx theme
 toml
 
-# required for autodoc
-numpy
-torch
+# dependencies for the tutorials
+chemfiles
+ase
+matplotlib
+skcosmo
+scikit-learn

docs/src/.gitignore

@@ -0,0 +1 @@
+examples

docs/src/conf.py

@@ -2,6 +2,7 @@
 import shutil
 import subprocess
 import sys
+from datetime import datetime
 
 import toml
 
@@ -11,9 +12,9 @@
 
 # -- Project information -----------------------------------------------------
 
-project = "equistore"
-copyright = "2022, Guillaume Fraux"
-author = "Guillaume Fraux"
+project = "Equistore"
+copyright = f"{datetime.now().date().year}, Equistore developers"
+author = "Equistore developers"
 
 
 def load_version_from_cargo_toml():
@@ -47,6 +48,10 @@ def build_doxygen_docs():
 build_doxygen_docs()
 
 
+def setup(app):
+ app.add_css_file("equistore.css")
+
+
 # -- General configuration ---------------------------------------------------
 
 needs_sphinx = "4.0.0"
@@ -56,6 +61,8 @@ def build_doxygen_docs():
 # ones.
 extensions = [
  "sphinx.ext.autodoc",
+ "sphinx.ext.intersphinx",
+ "sphinx_gallery.gen_gallery",
  "breathe",
 ]
 
@@ -73,6 +80,15 @@ def build_doxygen_docs():
 autodoc_typehints = "both"
 autodoc_typehints_format = "short"
 
+sphinx_gallery_conf = {
+ "filename_pattern": "/*",
+ "examples_dirs": ["../../python/examples"],
+ "gallery_dirs": ["examples"],
+ "min_reported_time": 60,
+ # Make the code snippet for equistore functions clickable
+ #"reference_url": {"equistore": None},
+ #"prefer_full_module": ["equistore"],
+}
 
 breathe_projects = {
  "equistore": os.path.join(ROOT, "docs", "build", "doxygen", "xml"),
@@ -82,6 +98,14 @@ def build_doxygen_docs():
  "h": "c",
 }
 
+intersphinx_mapping = {
+ "chemfiles": ("https://chemfiles.org/chemfiles.py/latest/", None),
+ "rascaline": ("https://luthaf.fr/rascaline/latest/", None),
+ "matplotlib": ("https://matplotlib.org/stable/", None),
+ "numpy": ("https://numpy.org/doc/stable/", None),
+ "python": ("https://docs.python.org/3", None),
+}
+
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages. See the documentation for
@@ -92,4 +116,22 @@ def build_doxygen_docs():
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-# html_static_path = ["_static"]
+html_static_path = [os.path.join(ROOT, "docs", "static")]
+
+html_theme_options = {
+ "footer_icons": [
+ {
+ "name": "GitHub",
+ "url": "https://github.com/lab-cosmo/equistore",
+ "html": "",
+ "class": "fa-brands fa-github fa-2x",
+ },
+ ],
+}
+
+# font-awesome logos (used in the footer)
+html_css_files = [
+ "https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/fontawesome.min.css",
+ "https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/solid.min.css",
+ "https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0/css/brands.min.css",
+]

docs/src/explanations/data.rst

@@ -0,0 +1,10 @@
+Where does the actual data come from?
+=====================================
+
+Equistore manages the metadata, where does the data come from. How does
+equistore deal with it and how to register new data origins in the python
+wrapper 
+
+(Python automagically transforms data to numpy.ndarray or a torch.tensor)
+Equistore has no idea about the data itself (only knows the pointer to data
+and operations you can perform on it - create, destroy move and reshape data)

docs/src/explanations/gradients.rst

@@ -0,0 +1,14 @@
+Gradients and how we manage them
+================================
+
+Gradient samples - "special" format
+
+first sample of gradients is "sample" that refers to the row in block.values
+that we are taking the gradient of.
+the other samples - what we are taking the gradient with respect to.
+Write what this entails -- block.gradients.sample (i A j) (pair feature i j A k)
+
+Cell gradients - Sample (i)
+components [[x y z ] [x y z]] (displacement matrix)
+
+Gradient wrt hypers

docs/src/explanations/index.rst

@@ -0,0 +1,15 @@
+.. _userdoc-explanations:
+
+Explanations
+============
+
+The explanation section discusses topics that broaden your knowledge of
+equistore. The theory behind the calculators and additional useful information
+are found here to give you more clarity and understanding of what equistore is
+all about.
+
+.. toctree::
+ :maxdepth: 2
+
+ data
+ gradients

docs/src/get-started/concepts.rst

@@ -0,0 +1,68 @@
+Core concepts of equistore
+##########################
+
+Let's have a brief look at the different kinds of data structures you might
+encounter when working with Equistore. For the purpose of making this
+explanation more tangible, let's consider we are working with a dataset of
+two water molecules [(O, H, H), (O,H,H)] and hyperparameters for computing
+the ACDCs to be (:math:`n_max = 3` and :math:`l_max = 2`).
+
+TensorMap
+---------
+
+A TensorMap object can be thought of as a container that holds blocks of data
+(and all the other following data structures) within it. It is the main object
+that you will encounter when using this library and can represent any data, for
+example the spherical expansion coefficients (ACDC of correlation order
+:math:`\nu = 1`) or the SOAP power spectrum(ACDC of correlation order
+:math:`\nu = 2` ) could be represented as a TensorMap. TensorMaps contain
+a list **TensorBlocks**, each addressed by a corresponding **key**. The keys
+reflect the nature of the atomic scale objects being represented and some
+crucial attributes to enable such sparsification.
+The closest analogy would be that of a *dict* object, that is similarly built
+on a collection of keys and corresponding values.
+It should be noted that the TensorMap is not just restricted to describe
+representations or descriptors, but could also very well be used to depict the
+targets of ML models, such as dipole moments or the effective single particle
+Hamiltonians. (see the how-to section for more details)
+
+For the example we are considering in this section, the TensorMap
+
+TensorBlock
+-----------
+
+A TensorBlock is the fundamental constituent of a TensorMap. Each block is
+addressed by a key of the TensorMap, so in the example above we would have
+six blocks for each of the keys.
+
+Each block in turn is associated with a data array with n-dimensions, each
+identified by a label. The first dimension refers to the *samples* that are
+tuples designating the data points that correspond to the key with which the
+block is associated.
+The TensorBlock associated with the key (*spherical_harmonics_l* = 1,
+*species_center* = 1) would have entries that contain information about the
+structure and index of the atoms that (here have species Hydrogen) thus yielding
+the samples to be [(0,1), (0,2), (1,1), (1,2)].
+
+The last dimension of the n-dimensional array indexes the properties or features
+of what we are describing in the TensorBlock. These also usually correspond to
+all the entries in the basis or :math: `<q|` when the object being represented
+in mathematically expressed as :math: `<q|f>`.
+For the given example, the property dimension would correspond to the radial
+channels or *n* going from 0 up to :math:`n_max`. [(0), (1), (2), (3)]
+
+All intermediate dimensions of the array are referred to as *components* and
+are used to describe vectorial or tensorial components of the data.
+
+A block can also contain gradients of the values with respect to a variety of
+parameters. More about this can be found in the **gradients** section.
+
+Labels
+------
+
+The set of keys is a equistore.Labels object, that also keeps track of the
+names that describe each index in the key.
+
+Each key of the TensorMap would be a tuple of the form (*spherical_harmonics_l*,
+*species_center*) and be linked to a corresponding TensorBlock. For this
+example, the list of keys would be [(0,1), (0,8), (1,1), (1,8), (2,1), (2,8)].

docs/src/get-started/equistore.rst

@@ -0,0 +1,28 @@
+What is equistore
+=================
+
+Equistore is a specialized data storage format suited to all your atomistic
+simulation needs and more. Equistore provides an accessible and understandable
+storage format for the data one comes across in atomistic machine learning.
+
+When working with large amounts of data, especially relating to atomistic
+simulations, one often needs access to the metadata such as the nature of
+the atomic scale objects being represented, various components seprated by
+symmetry, and .. to name a few. This metadata is implicit when storing this
+data as an array and it becomes increasingly painstaking to locate entries
+in the data corresponding to a specific selection of metadata (for example,
+imagine locating the gradients of the (nlm) component of the representation
+of atom *i* in structure *A* with respect to another atom *j*) with the size
+of the data (or the atomic entity).
+
+Another example arises when using equistore
+to compute atom-centered density correlation (ACDC) features, we can divide the
+descriptor data into blocks indexed by the chemical nature of the centers,
+behavior under symmetry operations (rotational and inversion), and the correlation
+order of the representation. Higher order features (in terms of correlations
+around the same center or including higher number of centers) can be computed
+by combining these blocks, a process that helps highlight their roles in model
+performance and tracks the information flow completely.
+This data that has been unraveled and stored into different blocks can be
+reassembled to a contiguous storage format, if desired, with a step-by-step
+control of data recombination.

docs/src/get-started/index.rst

@@ -1,39 +1,14 @@
+.. _userdoc-get-started:
+
 Getting started
 ===============
 
-.. TODO: expand this section
-
-.. _install-python-lib:
-
-Installing the Python library
------------------------------
-
-From source:
-
-.. code-block:: bash
-
- git clone https://github.com/lab-cosmo/equistore
- cd equistore
- pip install .
-
-Pre-built wheels:
-
-.. code-block:: bash
-
- pip install --extra-index-url https://luthaf.fr/temporary-wheels/ equistore
-
-.. _install-c-lib:
-
-Installing the C and C++ library
---------------------------------
-
-From source:
+The following sections describes how to install and start with using equistore.
 
-.. code-block:: bash
+.. toctree::
+ :maxdepth: 2
 
- git clone https://github.com/lab-cosmo/equistore
- cd equistore
- mkdir build && cd build
- cmake ..
- # configure cmake if needed
- cmake --build . --target install
+ equistore
+ concepts
+ installation
+ tutorials/index

docs/src/get-started/installation.rst

@@ -0,0 +1,65 @@
+Installation
+============
+
+You can install the latest version of equistore from one of the sources below.
+
+.. _install-python-lib:
+
+Installing the Python library
+-----------------------------
+
+From source:
+
+.. code-block:: bash
+
+ git clone https://github.com/lab-cosmo/equistore
+ cd equistore
+ pip install .
+
+Pre-built wheels:
+
+.. code-block:: bash
+
+ pip install --extra-index-url https://luthaf.fr/temporary-wheels/ equistore
+
+.. _install-c-lib:
+
+Installing the C and C++ library
+--------------------------------
+
+This installs a C-compatible shared library that can also be called from C++, as
+well as CMake files that can be used with ``find_package(equistore)``.
+
+.. code-block:: bash
+
+ git clone https://github.com/lab-cosmo/equistore
+ cd equistore
+ mkdir build && cd build
+ cmake <CMAKE_OPTIONS_HERE> ..
+ cmake --build . --target install
+
+The build and installation can be configures with a few cmake options, using
+``-D<OPTION>=<VALUE>`` on the cmake command line, or one of the cmake GUI
+(``cmake-gui`` or ``ccmake``). Here are the main configuration options:
+
++--------------------------+--------------------------------------------------------------------------------------+----------------+
+| Option | Description | Default |
++==========================+======================================================================================+================+
+| CMAKE_BUILD_TYPE | Type of build: debug or release | release |
++--------------------------+--------------------------------------------------------------------------------------+----------------+
+| CMAKE_INSTALL_PREFIX | Prefix in which the library will be installed | ``/usr/local`` |
++--------------------------+--------------------------------------------------------------------------------------+----------------+
+| INCLUDE_INSTALL_DIR | Path relative to ``CMAKE_INSTALL_PREFIX`` where the headers will be installed | ``include`` |
++--------------------------+--------------------------------------------------------------------------------------+----------------+
+| LIB_INSTALL_DIR | Path relative to ``CMAKE_INSTALL_PREFIX`` where the shared library will be installed | ``lib`` |
++--------------------------+--------------------------------------------------------------------------------------+----------------+
+
+Using the Rust library
+----------------------
+
+Add the following to your project ``Cargo.toml``
+
+.. code-block:: toml
+
+ [dependencies]
+ equistore = {git = "https://github.com/lab-cosmo/equistore.git"}

docs/src/get-started/tutorials/index.rst

@@ -0,0 +1,11 @@
+.. _userdoc-tutorials:
+
+Tutorials: using equistore from Python
+======================================
+
+The presented tutorials allow you to perform basic calculations in equistore.
+
+.. toctree::
+ :maxdepth: 1
+
+ ../../examples/first-tensormap