Add 8 initial templates

README.md

@@ -1,6 +1,43 @@
 # Dev Container Templates
 
-My own devcontainer templates with my own use in mind. Some of them will be really specific for what i like to use, while others might be more general purpose. Feel free to try to use them as a starting point or as templates for your own.
+My own Dev Container templates with my own uses in mind. Some of them will be very specific for what i like to use, while others might be more general purpose. Feel free to try to use them as a starting point or as templates for your own projects.
+
+## Template list
+
+Currently, it have the following templates available:
+ - **miniforge**: Miniforge3 official image
+ - **miniforge-dnd**: Miniforge3 official image, with Docker in Docker installed
+ - **python-utils**: Official Python image, with a few tools (poetry, ruff, mypy, MonkeyType and bandit) installed using pipx
+ - **python-dnd**: Same as python-utils, but also have Docker in Docker
+ - **pytorch-gpu**: Pytorch from the Nvidia NGC Catalog
+ - **tensorflow-gpu**: TensorFlow from the Nvidia NGC Catalog
+ - **local-colab**: The official Google Colab image
+ - **python-kaggle**: Official Python Kaggle image
+
+All templates create a non-root user (vscode), install zsh and sets it as the default shell, upgrade the packages and install git if it is not already available. The only exception is local-colab, which expects a root user.
+
+## Notes about image sizes
+
+**pytorch-gpu** and **tensorflow-gpu** are based on the images from the Nvidia NGC Catalog. These images are large **(~20 GB and ~14 GB respectively)**. They probably offer a great starting point for cloning machine learning repositories, but if storage is a constraint, consider using templates with smaller images (e.g. miniforge).
+
+**local-colab** and **python-kaggle** are based on Google Colab and Kaggle images respectively. These images are huge **(~28 GB and ~46 GB respectively)**. They can be useful for reproducing results from Colab and Kaggle Notebooks using your own hardware, which can be faster depending on you hardware. However, if storage is a constraint, consider using smaller images (e.g. miniforge).
+
+**python-utils** and **python-dnd** are based on the official Python images. These images occupy **~1.5 GB and ~2 GB respectively**. If you want slimmer images, consider using a [-slim tag](https://github.com/docker-library/docs/tree/master/python#pythonversion-slim), but be aware that
+
+> pip install may fail when installing a Python distribution package from a source distribution.
+
+**miniforge** and **miniforge-dnd** are based on the official Miniforge3 images. These images occupy **~0.5 GB and ~1 GB** respectively. They should allow you to install any of the required dependencies easily, including cuDNN for example.
+
+## Testing:
+
+To test all of the templates, you can use the `test.sh` script. Example:
+```
+./test.sh
+```
+Or to test a specific template, you can use the `test-template.sh` script with the name of the template as an argument. Example:
+```
+./test-template.sh miniforge
+```
 
 # Dev Container Templates: Self Authoring Guide
 

src/local-colab/.devcontainer/devcontainer.json

@@ -0,0 +1,81 @@
+{
+ // For quick reference: https://containers.dev/implementors/json_reference/
+ "name": "Local Colab",
+ // Same image from: https://research.google.com/colaboratory/local-runtimes.html
+ "image": "us-docker.pkg.dev/colab-images/public/runtime:latest",
+
+ // 1. Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // Use these ports if you want to connect to the local kernel through the colab service
+ // "forwardPorts": ["9000:8080"]
+
+ // 2. Use 'mounts' to make a list of local directories available inside the container. More info: https://code.visualstudio.com/remote/advancedcontainers/add-local-file-mount
+ // "mounts": [],
+
+ // 3. Use 'runArgs' to pass arguments to the container. 
+ // run the container with all GPUs
+ "runArgs": [
+ "--gpus",
+ "all"
+ ],
+
+ // 4. Features to add to the Dev Container. More info: https://containers.dev/implementors/features.
+ "features": {
+ // ZSH without OMZ, creates vscode user
+ "ghcr.io/devcontainers/features/common-utils:2": {
+ "installZsh": "true",
+ "configureZshAsDefaultShell": "true",
+ "installOhMyZsh": "false",
+ "installOhMyZshConfig": "false",
+ "username": "vscode",
+ "userUid": "1000",
+ "userGid": "1000",
+ "upgradePackages": "true"
+ },
+ // updates git
+ "ghcr.io/devcontainers/features/git:1": {
+ "version": "os-provided",
+ "ppa": "false"
+ }
+ },
+
+ // 5. Configure tool-specific properties.
+ "customizations": {
+ // Configure properties specific to VS Code.
+ "vscode": {
+ // Set *default* container specific settings.json values on container create.
+ "settings": {
+ "python.defaultInterpreterPath": "/usr/local/bin/python"
+ },
+ // install Jupyter vscode extension, allowing us to attach to the kernel and use it.
+ "extensions": [
+ "ms-toolsai.jupyter",
+ "ms-python.python"
+ ]
+ }
+ },
+
+ // 6. Set `remoteUser` to `vscode` to connect as vscode instead. More info: https://aka.ms/vscode-remote/containers/non-root.
+ // The container filesystem, and therefore the Jupyter Notebooks, expect
+ // a root user, so the default will be "root".
+ "remoteUser": "root"
+
+ // the following commands are related to container lifecylce. More info: https://containers.dev/implementors/json_reference/#lifecycle-scripts
+
+ // 7. Use 'initializeCommand' to run commands ON THE HOST before container is created or started.
+ // "initializeCommand": "",
+
+ // 8. Use 'onCreateCommand' to run commands INSIDE THE CONTAINER after it started for the first time.
+ // "onCreateCommand": "",
+
+ // 9. Use 'updateContentCommand' to run commands if the root filetree have any changes, during container creation.
+ // "updateContentCommand": "",
+
+ // 10. Use 'postCreateCommand' to run commands after the container is created.
+ // "postCreateCommand": "",
+
+ // 11. Use 'postStartCommand' to run a command each time the container starts successfully.
+ // "postStartCommand": "",
+
+ // 12. Use 'postAttachCommand' to run a command each time a tool attaches to the container successfully. 
+ // "postAttachCommand": "",
+}

src/local-colab/NOTES.md

@@ -0,0 +1,24 @@
+Template created using the instructions provided [here](https://research.google.com/colaboratory/local-runtimes.html)
+
+Ideally, it will allow to continue to run the same code locally without too much hassle.
+
+## Instructions
+
+1. Apply this template to the directory with the code you want to run
+2. Build the container and attach to it
+3. Open the Jupyter Notebook and select the kernel you want to use. There should be 2 listed, one for Python and one for R
+4. Run your code.
+
+## WARNING: Running untrusted code
+
+This container expects a root user to access most of the files, including simples things like the list of installed pip packages. **The default setting is to build the container and log in using the root user**. This makes it easier for attackers to get access to your system and potentially gain control over it. **So only run your own code, or code fromtrusted sources.**.
+
+You can opt out of using the root user by changing the `remoteUser` value to `"vscode"` in the `devcontainer.json`. But this will require you to run most of the commands with sudo first.
+
+## Attention: This Image is large (~28 GB)
+
+The compressed size listed inside the container registry is ~12 GB. During testing, the created images got to around ~28 GB.
+
+## Connecting runtime to Google Colab
+
+If you uncomment the `forwardPorts` session inside the `.devcontainer/devcontainer.json`, you can follow the instructions [here](https://research.google.com/colaboratory/local-runtimes.html) to connect your runtime to Google Colab and run code there, using your local machine.

src/local-colab/devcontainer-template.json

@@ -0,0 +1,12 @@
+{
+ "id": "local-colab",
+ "version": "0.0.1",
+ "name": "Local Colab (Google Colab Image)",
+ "description": "The official Google Colab runtime image, adapted to be run using Dev Containers.",
+ "documentationURL": "https://github.com/matheusfvesco/devcontainer-templates/tree/main/src/local-colab",
+ "licenseURL": "https://github.com/matheusfvesco/devcontainer-templates/blob/main/LICENSE",
+ "options": {},
+ "platforms": [
+ "Any"
+ ]
+}

src/miniforge-dnd/.devcontainer/Dockerfile

@@ -0,0 +1 @@
+FROM mcr.microsoft.com/devcontainers/python:${templateOption:imageVariant}

src/miniforge-dnd/.devcontainer/devcontainer.json

@@ -0,0 +1,77 @@
+{
+ // For quick reference: https://containers.dev/implementors/json_reference/
+ "name": "Miniforge3 Docker",
+ "image": "condaforge/miniforge3:${templateOption:imageVariant}",
+ //"build": {
+ // "context": "..",
+ // "dockerfile": "Dockerfile"
+ //},
+
+ // 1. Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // "forwardPorts": []
+
+ // 2. Use 'mounts' to make a list of local directories available inside the container. More info: https://code.visualstudio.com/remote/advancedcontainers/add-local-file-mount
+ // "mounts": [],
+
+ // 3. Use 'runArgs' to pass arguments to the container. 
+ // run the container with all GPUs
+ "runArgs": [
+ "--gpus",
+ "all"
+ ],
+
+ // 4. Features to add to the Dev Container. More info: https://containers.dev/implementors/features.
+ "features": {
+ // ZSH without OMZ
+ "ghcr.io/devcontainers/features/common-utils:2": {
+ "installZsh": "true",
+ "configureZshAsDefaultShell": "true",
+ "installOhMyZsh": "false",
+ "installOhMyZshConfig": "false",
+ "username": "vscode",
+ "userUid": "1000",
+ "userGid": "1000",
+ "upgradePackages": "true"
+ },
+ // git
+ "ghcr.io/devcontainers/features/git:1": {
+ "version": "os-provided",
+ "ppa": "false"
+ },
+ "ghcr.io/devcontainers/features/docker-in-docker": {}
+ },
+
+ // 5. Configure tool-specific properties.
+ "customizations": {
+ // Configure properties specific to VS Code.
+ "vscode": {
+ // Set *default* container specific settings.json values on container create.
+ "settings": {
+ "python.defaultInterpreterPath": "/opt/conda/bin/python"
+ }
+ }
+ },
+
+ // 6. Set `remoteUser` to `root` to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
+ "remoteUser": "vscode"
+
+ // the following commands are related to container lifecylce. More info: https://containers.dev/implementors/json_reference/#lifecycle-scripts
+
+ // 7. Use 'initializeCommand' to run commands ON THE HOST before container is created or started.
+ // "initializeCommand": "",
+
+ // 8. Use 'onCreateCommand' to run commands INSIDE THE CONTAINER after it started for the first time.
+ // "onCreateCommand": "",
+
+ // 9. Use 'updateContentCommand' to run commands if the root filetree have any changes, during container creation.
+ // "updateContentCommand": "",
+
+ // 10. Use 'postCreateCommand' to run commands after the container is created.
+ // "postCreateCommand": "pip3 install --user -r requirements.txt",
+
+ // 11. Use 'postStartCommand' to run a command each time the container starts successfully.
+ // "postStartCommand": "",
+
+ // 12. Use 'postAttachCommand' to run a command each time a tool attaches to the container successfully. 
+ // "postAttachCommand": "",
+}

src/miniforge-dnd/NOTES.md

@@ -0,0 +1,10 @@
+This container have miniforge installed on `/opt/conda` and is run with `--gpus all`. This makes installing tools like CUDA, Cuda Toolkit, cuDNN etc. much easier.
+
+## Tip 1: Keep the Base environment Clean
+
+Installing most things inside base is an option, but can sometimes slow down during package solving. [It is recommended to keep base clean of all dependencies except for those related to conda](https://github.com/conda/conda/issues/11919#issuecomment-1283923009). Create a new environent for your project.
+
+## Tip 2: Use Libmamba
+
+[Use Mamba](https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community), a faster solver for conda.
+Miniconda should already have libmamba installed.

src/miniforge-dnd/devcontainer-template.json

@@ -0,0 +1,35 @@
+{
+ "id": "miniforge-dnd",
+ "version": "0.0.1",
+ "name": "Miniforge3 Docker (DnD and GPU)",
+ "description": "A Python template with miniforge3 installed and --gpus all passed to the container, making it easy to develop any project.",
+ "documentationURL": "https://github.com/matheusfvesco/devcontainer-templates/tree/main/src/miniforge-dnd",
+ "licenseURL": "https://github.com/matheusfvesco/devcontainer-templates/blob/main/LICENSE",
+ "options": {
+ "imageVariant": {
+ "type": "string",
+ "description": "The miniforge version (MAJOR.MINOR.PATCHES):",
+ "proposals": [
+ "latest",
+ "24.3.0-0",
+ "24.1.2-0",
+ "23.11.0-0",
+ "23.3.1-1",
+ "23.3.1-0",
+ "23.1.0-4",
+ "23.1.0-3",
+ "23.1.0-2",
+ "23.1.0-1",
+ "22.11.1-4",
+ "22.9.0-3",
+ "22.9.0-2",
+ "22.9.0-1",
+ "22.9.0-0"
+ ],
+ "default": "24.3.0-0"
+ }
+ },
+ "platforms": [
+ "Any"
+ ]
+}

src/miniforge/.devcontainer/Dockerfile

@@ -0,0 +1 @@
+FROM mcr.microsoft.com/devcontainers/python:${templateOption:imageVariant}

src/miniforge/.devcontainer/devcontainer.json

@@ -0,0 +1,76 @@
+{
+ // For quick reference: https://containers.dev/implementors/json_reference/
+ "name": "Miniforge3",
+ "image": "condaforge/miniforge3:${templateOption:imageVariant}",
+ //"build": {
+ // "context": "..",
+ // "dockerfile": "Dockerfile"
+ //},
+
+ // 1. Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // "forwardPorts": []
+
+ // 2. Use 'mounts' to make a list of local directories available inside the container. More info: https://code.visualstudio.com/remote/advancedcontainers/add-local-file-mount
+ // "mounts": [],
+
+ // 3. Use 'runArgs' to pass arguments to the container. 
+ // run the container with all GPUs
+ "runArgs": [
+ "--gpus",
+ "all"
+ ],
+
+ // 4. Features to add to the Dev Container. More info: https://containers.dev/implementors/features.
+ "features": {
+ // ZSH without OMZ
+ "ghcr.io/devcontainers/features/common-utils:2": {
+ "installZsh": "true",
+ "configureZshAsDefaultShell": "true",
+ "installOhMyZsh": "false",
+ "installOhMyZshConfig": "false",
+ "username": "vscode",
+ "userUid": "1000",
+ "userGid": "1000",
+ "upgradePackages": "true"
+ },
+ // git
+ "ghcr.io/devcontainers/features/git:1": {
+ "version": "os-provided",
+ "ppa": "false"
+ }
+ },
+
+ // 5. Configure tool-specific properties.
+ "customizations": {
+ // Configure properties specific to VS Code.
+ "vscode": {
+ // Set *default* container specific settings.json values on container create.
+ "settings": {
+ "python.defaultInterpreterPath": "/opt/conda/bin/python"
+ }
+ }
+ },
+
+ // 6. Set `remoteUser` to `root` to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
+ "remoteUser": "vscode"
+
+ // the following commands are related to container lifecylce. More info: https://containers.dev/implementors/json_reference/#lifecycle-scripts
+
+ // 7. Use 'initializeCommand' to run commands ON THE HOST before container is created or started.
+ // "initializeCommand": "",
+
+ // 8. Use 'onCreateCommand' to run commands INSIDE THE CONTAINER after it started for the first time.
+ // "onCreateCommand": "",
+
+ // 9. Use 'updateContentCommand' to run commands if the root filetree have any changes, during container creation.
+ // "updateContentCommand": "",
+
+ // 10. Use 'postCreateCommand' to run commands after the container is created.
+ // "postCreateCommand": "pip3 install --user -r requirements.txt",
+
+ // 11. Use 'postStartCommand' to run a command each time the container starts successfully.
+ // "postStartCommand": "",
+
+ // 12. Use 'postAttachCommand' to run a command each time a tool attaches to the container successfully. 
+ // "postAttachCommand": "",
+}

src/miniforge/NOTES.md

@@ -0,0 +1,10 @@
+This container have miniforge installed on `/opt/conda` and is run with `--gpus all`. This makes installing tools like CUDA, Cuda Toolkit, cuDNN etc. much easier. It also have Docker Inside Docker, which can be used to develop dockerized applications, or to easily get databases up and running for your application.
+
+## Tip 1: Keep the Base environment Clean
+
+Installing most things inside base is an option, but can sometimes slow down during package solving. [It is recommended to keep base clean of all dependencies except for those related to conda](https://github.com/conda/conda/issues/11919#issuecomment-1283923009).
+
+## Tip 2: Use Libmamba
+
+[Use Mamba](https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community), a faster solver for conda.
+Miniconda should already have libmamba installed.