start adding docs, license file

.github/workflows/docs.yml

@@ -0,0 +1,24 @@
+name: Deploy docs
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  deploy-docs:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+      - name: Install dependencies
+        run: python -m pip install -r docs-requirements.txt
+      - name: Build and deploy docs
+        run: mkdocs gh-deploy --clean --verbose

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 The University of Auckland
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

docs-requirements.txt

@@ -0,0 +1,2 @@
+mkdocs
+mkdocs-material

docs/apps/example-conda-jupyterlab-ml101.md

@@ -0,0 +1,3 @@
+# Example: conda based JupyterLab app for ML101
+
+TODO...

docs/apps/example-rstudio-rna-seq.md

@@ -0,0 +1,5 @@
+# RStudio based app for RNA-Seq
+
+## Overview
+
+App developed for the [RNA-Seq](https://genomicsaotearoa.github.io/RNA-seq-workshop/) workshop using RStudio.

docs/apps/example-venv-jupyterlab-ml102.md

@@ -0,0 +1,3 @@
+# Example: venv based JupyterLab app for ML102
+
+TODO...

docs/apps/index.md

@@ -0,0 +1,21 @@
+# Apps
+
+## Overview
+
+Open OnDemand app development is described in detail in their [documentation](https://osc.github.io/ood-documentation/latest/how-tos/app-development.html).
+
+In particular, the training environment uses a kubernetes cluster for running the apps, so the apps are developed similarly to the kubernetes examples on the Open OnDemand website:
+
+- [Add a Jupyter App on a Kubernetes Cluster](https://osc.github.io/ood-documentation/latest/tutorials/tutorials-interactive-apps/k8s-jupyter.html)
+- [Add a Jupyter App on a Kubernetes Cluster that behaves like HPC compute](https://osc.github.io/ood-documentation/latest/tutorials/tutorials-interactive-apps/k8s-like-hpc-jupyter.html)
+
+On the following pages are some examples of apps that have been developed to run on the training environment.
+
+## Key points
+
+- Apps are created as docker images
+- Apps are set up as LDAP clients so *training* and *trainer* users are identified correctly within the container
+    * the nslcd socket from the kubernetes worker node is bound into the container at the correct location
+    * the */etc/nsswitch.conf* file from the kubernetes worked node is bound into the container
+    * LDAP client packages are installed inside the container (nslcd, etc)
+- All home directories are bound into the container (so *trainer* users can access *training* users' homes from within the apps if needed)

docs/deployment.md

@@ -0,0 +1,5 @@
+# Deployment
+
+TODO:
+
+- process, configuring (num users, etc)

docs/index.md

@@ -0,0 +1,32 @@
+# Overview
+
+NeSI ephemeral training environments, running on Flexi HPC, deployed using Terraform and Ansible and using [Open OnDemand](https://osc.github.io/ood-documentation/latest/index.html) as the user interface.
+
+## Architecture 
+
+The training environment consists of multiple VMs:
+
+- web node VM
+    * running the Open OnDemand web node software
+    * LDAP client
+- services node VM
+    * LDAP server
+    * Keycloak
+    * NFS server sharing training user home directories
+- kubernetes cluster
+    * the users' Open OnDemand apps run here (JupyterLab, RStudio, etc.)
+    * each node in the cluster is an LDAP client
+
+## Apps
+
+Open OnDemand [interactive applications](https://osc.github.io/ood-documentation/latest/index.html), such as JupyterLab and RStudio, are used for the training courses. More information about apps can be found [here](/apps).
+
+## User accounts
+
+Within the training environment we create two different types of user accounts, *training* users and *trainer* users, with randomly generated passwords.
+
+- *training* accounts
+    * isolated accounts that can only access their own home directory
+- *trainer* accounts
+    * have read-only access to *training* user home directories, to monitor progress and assist
+    * can use the browser based terminal from the OnDemand web interface

docs/notes-for-trainers.md

@@ -0,0 +1,28 @@
+# Notes for trainers
+
+TODO:
+
+- notes for trainers (where to find password), recommended browsers, etc
+
+## Recommended browsers
+
+Following upstream Open OnDemand [documentation](https://osc.github.io/ood-documentation/latest/requirements.html#browser-requirements) we highly recommend the use of Chrome, Firefox or Edge.
+
+Other browsers and extensions (ad blockers, etc) could cause issues. For example, we have observed problems with the DuckDuckGo browser, which blocked the password from being passed transparently to JupyterLab, causing an "Invalid credentials" error.
+
+![jupyter credentials error](jupyter-browser-error.png)
+
+## Session limits
+
+Each user (both *trainer* and *training* users) are limited to one running session (app) at a time, to ensure there are enough resources for everyone to run something. To restart an app, or stop and app so that you can start another one, you should:
+
+- browse to the "My Interactive Sessions" tab
+- click "Delete" on the running session
+- **important:** wait for approx one minute for the underlying pod to get properly deleted by the kubernetes cluster (otherwise you will get an error at the next step)
+- launch the new app
+
+## Browser based terminal app
+
+![webnode shell access](webnode-shell-access.png)
+
+The browser based terminal in the OnDemand web interface runs on the webnode and can be used to monitor progress and assist *training* users from outside the apps. These terminal sessions run on the OnDemand web node and do not have any resource restrictions, so nothing resource intensive should be run there. Only *trainer* users can use this feature; if *training* users try to use it they will get an error.

mkdocs.yml

@@ -0,0 +1,14 @@
+site_name: NeSI Training Environment
+
+nav:
+  - Overview: 'index.md'
+  - Apps:
+    - Overview: 'apps/index.md'
+    - "Example: conda based JupyterLab app for ML101": 'apps/example-conda-jupyterlab-ml101.md'
+    - "Example: venv based JupyterLab app for ML102": 'apps/example-venv-jupyterlab-ml102.md'
+    - "Example: RStudio based app for RNA-Seq": 'apps/example-rstudio-rna-seq.md'
+  - Deployment: 'deployment.md'
+  - Notes for trainers: 'notes-for-trainers.md'
+
+theme:
+  name: "material"