feat: canonical k8s integration (#219)

* Fix MLflow deployment for Canonical k8s (#195)

* Fix MLflow deployment for Canonical k8s

* Fix init container name

* Fix shared folder on Canonical k8s (#196)

* feat: run integration tests on canonical k8s (#203)

* feat: run integration tests on canonical k8s

* fix: gpu tests to run on Canonical k8s (#204)

* fix: run gpu ci on caonical k8s

* feat: update docs for canonical k8s (#215)

* feat: update docs for canonical k8s
* Angel's review

---------

Co-authored-by: afgambin <angel.fernandez@canonical.com>

* fix: use security context for the mounted volumens (#214)

* fix: store dss logs in snap common folder (#218)

* fix: store logs in snap common folder

---------

Co-authored-by: deusebio <edeusebio85@gmail.com>

---------

Co-authored-by: afgambin <angel.fernandez@canonical.com>
Co-authored-by: deusebio <edeusebio85@gmail.com>

Author: 3 people

.github/workflows/aws-integration-gpu.yaml

@@ -40,21 +40,48 @@ jobs:
     - name: Checkout repository
       uses: actions/checkout@v4
 
-    # Issue: https://github.com/canonical/data-science-stack/issues/116
-    - name: Setup operator environment
-      run: sudo -E bash .github/workflows/setup_environment.sh
-      env:
-        MICROK8S_CHANNEL: 1.28/stable
+    - name: Install python
+      run: |
+        sudo add-apt-repository ppa:deadsnakes/ppa -y
+        sudo apt update -y
+        VERSION=3.10
+        sudo apt install python3.10 python3.10-distutils python3.10-venv -y
+        wget https://bootstrap.pypa.io/get-pip.py
+        python3.10 get-pip.py
+        python3.10 -m pip install tox
+        rm get-pip.py
+    
+    - name: Install and setup Canonical k8s
+      run: |        
+        sudo snap install kubectl --classic
+        sudo snap install k8s --classic --channel=1.32-classic/stable
+        sudo k8s bootstrap
+        sudo k8s enable local-storage
+        mkdir -p ~/.kube
+        sudo k8s config > ~/.kube/config
+        sudo chown $(id -u):$(id -g) ~/.kube/config
+    
+    - name: Enable NVIDIA operator
+      run: |
+        export KUBECONFIG=~/.kube/config
+        curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 \
+            && chmod 700 get_helm.sh \
+            && ./get_helm.sh
+
+        helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \
+            && helm repo update
+
+        helm install --wait --generate-name -n gpu-operator --create-namespace nvidia/gpu-operator
 
-    - name: Setup microk8s
-      run: sudo -E bash .github/workflows/configure_microk8s.sh
-      env:
-        MICROK8S_ADDONS: "storage dns rbac gpu"
+        # Wait until the GPU operator validations pass
+        while ! kubectl logs -n gpu-operator -l app=nvidia-operator-validator | grep "all validations are successful"; do
+            echo "Waiting for GPU operator validations to pass..."
+            sleep 5
+        done
 
-    - name: Run tests as root
+    - name: Run tests
       run: |
-        sudo snap alias microk8s.kubectl kubectl
-        tox -e integration-gpu -- -vv -s
+        sudo tox -e integration-gpu -- -vv -s
 
   stop-runner:
     name: Stop self-hosted EC2 runner

.github/workflows/build_and_test_snap.yaml

@@ -22,7 +22,7 @@ jobs:
     - name: Checkout repository
       uses: actions/checkout@v4
 
-    - uses: snapcore/action-build@v1.2.0
+    - uses: snapcore/action-build@v1.3.0
       id: snapcraft
       with:
         # Use a deterministic file name, so we can easily reference it later

.github/workflows/configure_microk8s.sh

@@ -36,8 +36,6 @@ jobs:
     # is resolved and applied to all ROCKs repositories
     - name: Install python version from input
       run: |
-        # TODO: remove when fixed: https://github.com/microsoft/linux-package-repositories/issues/130#issuecomment-2074645171
-        sudo rm /etc/apt/sources.list.d/microsoft-prod.list
         sudo add-apt-repository ppa:deadsnakes/ppa -y
         sudo apt update -y
         VERSION=3.10
@@ -47,11 +45,20 @@ jobs:
         python3.10 -m pip install tox
         rm get-pip.py
     
-    - uses: balchua/microk8s-actions@v0.3.2
-      with:
-        channel: '1.28/stable'
-        addons: '["hostpath-storage"]'
+    - name: Install dependencies
+      run: |
+        # Removing docker as it is blocking canonical k8s bootstrap
+        # Based on this guide https://documentation.ubuntu.com/canonical-kubernetes/latest/snap/howto/install/dev-env/#containerd-conflicts
+        sudo apt-get remove -y docker-ce docker-ce-cli containerd.io
+        sudo rm -rf /run/containerd
+        
+        sudo snap install kubectl --classic
+        sudo snap install k8s --classic --channel=1.32-classic/stable
+        sudo k8s bootstrap
+        sudo k8s enable local-storage
+        mkdir -p ~/.kube
+        sudo k8s config > ~/.kube/config
 
-    - name: Install library
-      run: sg microk8s -c "tox -vve integration"
+    - name: Run tests
+      run: tox -vve integration
 

.github/workflows/setup_environment.sh

@@ -1,10 +1,10 @@
 DSS architecture
 ================
 
-This guide provides an overview of the Data science stack (DSS) architecture, its main components and their interactions. 
+This guide provides an overview of the Data Science Stack (DSS) architecture, its main components, and their interactions. 
 
 DSS is a ready-to-run environment for Machine Learning (ML) and Data Science (DS). 
-It's built on open-source tooling, including `MicroK8s`_, JupyterLab and `MLflow <https://ubuntu.com/blog/what-is-mlflow>`_.
+It's built on open-source tooling, including `Canonical K8s`_, JupyterLab, and `MLflow <https://ubuntu.com/blog/what-is-mlflow>`_.
 
 DSS is distributed as a `snap`_ and usable on any Ubuntu workstation. 
 This provides robust security management and user-friendly version control, enabling seamless updates and auto-rollback in case of failure. 
@@ -52,22 +52,22 @@ ML tools
 
 DSS includes:
 
-* Jupyter Notebooks: Open source environment that provides a flexible interface to organise DS projects and ML workloads. 
-* MLflow: Open source platform for managing the ML life cycle, including experiment tracking and model registry.
-* ML frameworks: DSS comes by default with PyTorch and Tensorflow. Users can manually add other frameworks, depending on their needs and use cases.
+* Jupyter Notebooks: Open-source environment that provides a flexible interface to organise DS projects and ML workloads. 
+* MLflow: Open-source platform for managing the ML life cycle, including experiment tracking and model registry.
+* ML frameworks: DSS comes by default with PyTorch and TensorFlow. Users can manually add other frameworks, depending on their needs and use cases.
 
 Jupyter Notebooks
 ^^^^^^^^^^^^^^^^^
 
-A `Jupyter Notebook <Jupyter Notebooks_>`_ is essentially a `Kubernetes deployment <Pod_>`_, also known as `Pod`, running a Docker image with Jupyter Lab and a dedicated ML framework, such as Pytorch or Tensorflow.
+A `Jupyter Notebook <Jupyter Notebooks_>`_ is essentially a `Kubernetes deployment <Pod_>`_, also known as `Pod`, running a Docker image with Jupyter Lab and a dedicated ML framework, such as PyTorch or TensorFlow.
 For each Jupyter Notebook, DSS mounts a `Hostpath <Microk8s hostpath docs_>`_ directory-backed persistent volume to the data directory. 
 All Jupyter Notebooks share the same persistent volume, allowing them to exchange data seamlessly. 
 The full path to that persistent volume is `/home/jovyan/shared`.
 
 MLflow
 ^^^^^^
 
-`MLflow <https://ubuntu.com/blog/what-is-mlflow>`_ operates in `local mode <https://mlflow.org/docs/latest/tracking.html#other-configuration-with-mlflow-tracking-server>`_, 
+`MLflow <https://ubuntu.com/blog/what-is-mlflow>`_ operates in `local mode <https://mlflow.org/docs/latest/tracking/#other-tracking-setup>`_, 
 meaning that metadata and artefacts are, by default, stored in a local directory.
 
 This local directory is backed by a persistent volume, mounted to a Hostpath directory of the MLflow Pod.
@@ -79,27 +79,22 @@ Orchestration
 ~~~~~~~~~~~~~
 
 DSS requires a container orchestration solution. 
-DSS relies on `MicroK8s`_, a lightweight Kubernetes distribution.
+DSS relies on `Canonical K8s`_, a lightweight Kubernetes distribution.
 
-Therefore, MicroK8s needs to be deployed before installing DSS on the host machine. 
-It must be configured with the storage add-on. 
-This is required to use Hostpath storage in the cluster. 
-See :ref:`set_microk8s` to learn how to install MicroK8s.
+Therefore, Canonical K8s needs to be deployed before installing DSS on the host machine. 
+It must be configured with local storage support to handle persistent volumes used by DSS.
 
 .. _gpu_support:
 
 GPU support
 ^^^^^^^^^^^
 
 DSS can run with or without the use of GPUs.
-If needed, MicroK8s can be configured with the desired `GPU add-on <https://microk8s.io/docs/addon-gpu>`_. 
-
-DSS is designed to support the deployment of containerised GPU workloads on NVIDIA GPUs. 
-MicroK8s simplifies the GPU access and usage through the `NVIDIA GPU Operator <NVIDIA Operator_>`_. 
+If needed, follow `NVIDIA GPU Operator <https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/getting-started.html>`_ for deployment details.
 
 DSS does not automatically install the tools and libraries required for running GPU workloads.
-To do so, it relies on MicroK8s for the required operating-system drivers.
-It also relies on the chosen image, for example, CUDA when working with NVIDIA GPUs.
+It relies on Canonical K8s for the required operating-system drivers.
+It also depends on the chosen image, for example, CUDA when working with NVIDIA GPUs.
 
 .. caution::
    GPUs from other silicon vendors rather than NVIDIA can be configured. However, its functionality is not guaranteed.
@@ -108,16 +103,16 @@ Storage
 ^^^^^^^
 
 DSS expects a default `storage class <https://kubernetes.io/docs/concepts/storage/storage-classes/>`_ in the Kubernetes deployment, which is used to persist Jupyter Notebooks and MLflow artefacts.   
-In MicroK8s, the Hostpath storage add-on is chosen, used to provision Kubernetes' *PersistentVolumeClaims* (`PVCs <https://kubernetes.io/docs/concepts/storage/persistent-volumes/>`_). 
+In Canonical K8s, a local storage class should be configured to provision Kubernetes' *PersistentVolumeClaims* (`PVCs <https://kubernetes.io/docs/concepts/storage/persistent-volumes/>`_). 
 
 A shared PVC is used across all Jupyter Notebooks to share and persist data. 
 MLflow also uses its dedicated PVC to store the logged artefacts.
 This is the DSS default storage configuration and cannot be altered.
 
-This choice ensures that all storage is backed up on the host machine in the event of MicroK8s restarts.
+This choice ensures that all storage is backed up on the host machine in the event of cluster restarts.
 
 .. note::
-   By default, you can access the DSS storage anytime under your local directory `/var/snap/microk8s/common/default-storage`. 
+   By default, you can access the DSS storage anytime under your local directory `/var/snap/k8s/common/default-storage`. 
 
 The following diagram summarises the DSS storage:
 
@@ -132,7 +127,7 @@ The following diagram summarises the DSS storage:
 Operating system
 ~~~~~~~~~~~~~~~~
 
-DSS is native on Ubuntu, being developed, tested and validated on it. 
+DSS is native on Ubuntu, being developed, tested, and validated on it. 
 Moreover, the solution can be used on any Linux distribution.
 
 Namespace configuration
@@ -147,8 +142,7 @@ This includes the GPU Operator for managing access and usage.
 Accessibility
 -------------
 
-Jupyter Notebooks and MLflow can be accessed from a web browser through the Pod IP that is given access through MicroK8s.
+Jupyter Notebooks and MLflow can be accessed from a web browser through the Pod IP that is given access through Canonical K8s.
 See :ref:`access_notebook` and :ref:`access_mlflow` for more details.
 
-
-
+.. _Canonical K8s: https://snapcraft.io/k8s

.github/workflows/tests.yaml

@@ -8,10 +8,10 @@ This guide describes how to manage Data Science Stack (DSS).
 DSS is a Command Line Interface (CLI)-based environment and distributed as a `snap`_.
 
 Install
---------
+-------
 
 .. note::
-   To install DSS, ensure you have previously installed `Snap`_ and `MicroK8s`_.
+   To install DSS, ensure you have previously installed `Snap`_ and `Canonical K8s`_.
 
 You can install DSS using ``snap`` as follows:
 
@@ -26,20 +26,20 @@ Then, you can run the DSS CLI with:
     dss
 
 Initialise
------------
+----------
 
 You can initialise DSS through ``dss initialize``.
 This command:
 
-* Stores credentials for the MicroK8s cluster.
+* Stores credentials for the Canonical K8s cluster.
 * Allocates storage for your DSS Jupyter Notebooks.
 * Deploys an `MLflow <MLflow Docs_>`_ model registry.
 
 .. code-block:: shell
 
-    dss initialize --kubeconfig "$(sudo microk8s config)"
+    dss initialize --kubeconfig "$(sudo k8s config)"
 
-The ``--kubeconfig`` option is used to provide your MicroK8s cluster's kubeconfig.
+The ``--kubeconfig`` option is used to provide your Canonical K8s cluster's kubeconfig.
 
 .. note::
    Note the use of quotes for the ``--kubeconfig`` option. Without them, the content may be interpreted by your shell.
@@ -61,9 +61,9 @@ You should expect an output like this:
       dss create my-notebook --image=kubeflownotebookswg/jupyter-scipy:v1.8.0
 
 Remove
--------
+------
 
-You can remove DSS from your MicroK8s cluster through ``dss purge``. 
+You can remove DSS from your Canonical K8s cluster through ``dss purge``. 
 This command purges all the DSS components, including:
 
 * All Jupyter Notebooks.
@@ -72,8 +72,8 @@ This command purges all the DSS components, including:
 
 .. note::
 
-    This action removes the components of the DSS environment, but it does not remove the DSS CLI or your MicroK8s cluster.  
-    To remove those, `delete their snaps <https://snapcraft.io/docs/quickstart-tour>`_.
+    This action removes the components of the DSS environment, but it does not remove the DSS CLI or your Canonical K8s cluster.  
+    To remove those, `delete their snaps <https://snapcraft.io/docs/get-started>`_.
 
 .. code-block:: bash
 
@@ -91,7 +91,7 @@ You should expect an output like this:
     Success: All DSS components and notebooks purged successfully from the Kubernetes cluster.
 
 Get status
------------
+----------
 
 You can check the DSS status through ``dss status``. 
 This command provides a quick way to check the status of your DSS environment, including the MLflow status and whether a GPU is detected in your environment.
@@ -109,7 +109,7 @@ If you already have a DSS environment running and no GPU available, the expected
     GPU acceleration: Disabled
 
 List commands
---------------
+-------------
 
 You can get the list of available commands for DSS through the ``dss`` command with the ``--help`` option:
 
@@ -134,12 +134,11 @@ You should expect an output like this:
     list        Lists all created notebooks in the DSS environment.
     logs        Prints the logs for the specified notebook or DSS component.
     purge       Removes all notebooks and DSS components.
-    remove      Remove a Jupter Notebook in DSS with the name NAME.
+    remove      Remove a Jupyter Notebook in DSS with the name NAME.
     start       Starts a stopped notebook in the DSS environment.
     status      Checks the status of key components within the DSS...
     stop        Stops a running notebook in the DSS environment.
 
-    
 **Get details about a specific command**:
 
 To see the usage and options of a DSS command, run ``dss <command>`` with the ``--help`` option.
@@ -174,4 +173,4 @@ See also
 --------
 
 * To learn how to manage your Jupyter Notebooks, check :ref:`manage_notebooks`. 
-* If you are interested in managing MLflow within your DSS environment, see :ref:`manage_MLflow`.
+* If you are interested in managing MLflow within your DSS environment, see :ref:`manage_MLflow`.