Simplify setup of Docker-based machines

[jemrobinson]

💻 System information

• Data Safe Haven version: 2.0.0-beta

🚫 Problematic behaviour

We currently run full Ubuntu VMs to run applications in Docker containers. We should consider alternatives

🚂 Possible workarounds, remediations or solutions

CoreOS

There are examples of how to configure machines that only run Docker images using CoreOS on Azure. This would reduce the attack surface of these machines compared to an Ubuntu VM as they have less functionality.

Azure container instances

Azure container instances have been tested and found to work well in the v4prototype migration testing.

[JimMadge]

CoreOS is now end of life. It has been (more or less) replaced by Fedora CoreOS after being bought by Red Hat.

Unfortunately, there isn't a Fedora CoreOS image in the Azure market place yet. Although they do provide an image that could be uploaded to Azure.

[darenasc]

RancherOS could be an option as a distribution for containers.

[JimMadge]

@darenasc Have you had any experience with RancherOS? It's one I've seen as an option on cloud platforms but I've not heard much about it or anyone using it so I've always been (probably unfairly) suspicious of it.

[darenasc]

Sorry, I haven't used RancherOS (but I'd like to) I've only seen it on workshops and talks about it where it seems to be good for monitoring the containers as part of the SO. In Azure I have used its Container Instances (ACI) service to deploy ML models, and it is simple to deploy and monitor images and containers, there is also Azure Kubernetes Services (AKS) but I haven't use it for projects.

[JimMadge]

Exploring Fedora CoreOS a little more,

• It is possible and not too difficult to automate getting an image on Azure. However, unless there is a way to download directly to a storage account it requires using your local machine as a middle-man for the image. The process is,
  • Download image from Fedora
  • Put in blob storage
  • Build Azure image object from image in blob storage
• Configuration is simple, but adds one dependency (fcct)
  • Write a human-readable YAML configuration
  • Convert to a JSON format using fcct
• Creating a VM is straightforward, you pass the JSON configuration file as custom-data
• Operations on the CoreOS machine may be executed as systemd units.
• The preferred container manager is podman. This should be more secure than docker as it can be run as a non-privileged user, but this does mean there are some subtle difference to docker.

[JimMadge]

HackMD running on Fedora CoreOS in Azure

Configuration file

variant: fcos

version: 1.0.0

passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAA....

systemd:
  units:
    - name: postgres.service
      enabled: true
      contents: |
        [Unit]
        Description=Deploy a postgres container
        After=network-online.target
        Wants=network-online.target

        [Service]
        TimeoutStartSec=0
        ExecStartPre=-/bin/podman kill database
        ExecStartPre=-/bin/podman rm database
        ExecStartPre=/bin/podman pull postgres:11.6-alpine
        ExecStart=/bin/podman run --name database --pod new:hackmdpod -p 80:3000 -e POSTGRES_USER=hackmd -e POSTGRES_PASSWORD=change_password -e POSTGRES_DB=hackmd postgres:11.6-alpine

        [Install]
        WantedBy=multi-user.target
    - name: hackmd.service
      enabled: true
      contents: |
        [Unit]
        Description=Deploy a HackMD container
        After=network-online.target
        Wants=network-online.target
        Requires=postgres.service
        After=postgres.service

        [Service]
        TimeoutStartSec=0
        ExecStartPre=-/bin/podman kill hackmd
        ExecStartPre=-/bin/podman rm hackmd
        ExecStartPre=/bin/podman pull hackmdio/hackmd:2.1.0
        ExecStart=/bin/podman run --name hackmd --pod hackmdpod -e CMD_DB_URL=postgres://hackmd:change_password@localhost/hackmd -e CMD_USECDN=false hackmdio/hackmd:2.1.0

        [Install]
        WantedBy=multi-user.target

Notice that I use a 'pod' to allow the two containers to be connected via localhost. Being unable to use a docker (style) network is one of the restrictions of rootless containers.