runner --driver=github Docker-in-Docker

[casperdcl]

Docker-in-Docker (probably) doesn't work with GitHub?

(P.S. we recommend --docker-volumes=/var/run/docker.sock:/var/run/docker.sock for GitLab)

jobs:
  deploy:
    steps:
    - run: cml runner --cloud=aws --labels=cml-aws
  train:
    needs: deploy
    runs-on: [self-hosted, cml-aws]
    container:
      image: docker://<some_container>
    steps:
    - uses: [self-hosted, cml-aws]
    - run: docker run ...

1. Have we tried using docker:dind for <some_container>?
   • Do we need container.volumes: /var/run/docker.sock:/var/run/docker.sock?
2. Have we confirmed not using a container: works (i.e. Docker-on-baremetal)?

[0x2b3bfa0]

Working

GitHub takes care of mounting any volumes it needs for running docker commands inside container jobs.

container: docker:dind

on: workflow_dispatch     
jobs:
  example:
    runs-on: ubuntu-latest
    container: docker:dind
    steps:
    - run: echo FROM python > Dockerfile
    - run: docker build .

container: docker

on: workflow_dispatch     
jobs:
  example:
    runs-on: ubuntu-latest
    container: docker
    steps:
    - run: echo FROM python > Dockerfile
    - run: docker build .

container: alpine + apk add docker

on: workflow_dispatch     
jobs:
  example:
    runs-on: ubuntu-latest
    container: alpine
    steps:
    - run: apk add docker
    - run: echo FROM python > Dockerfile
    - run: docker build .

container: null

on: workflow_dispatch     
jobs:
  example:
    runs-on: ubuntu-latest
    steps:
    - run: echo FROM python > Dockerfile
    - run: docker build .

Not Working

Containerized self–hosted runners don't admit container steps nor container actions.

on: workflow_dispatch     
jobs:
  example:
    runs-on: self-hosted # containerized runner
    container: alpine
    steps:
    - run: date

Error: Container feature is not supported when runner is already running inside container.

• Provide runner as a Docker Image actions/runner#367
• Support Runner inside of Docker Container actions/runner#406
• 👨🏼‍💻 actions/runner@698d3a2/src/Runner.Worker/ContainerOperationProvider.cs:67-71

📖 As long as users don't set the container option nor use container actions, containerized self–hosted runners will work as expected.

🧙🏼 It's also possible to run docker commands inside a containerized self–hosted runner, as long as the underlying container has the docker command installed and has /var/run/docker.sock mounted to the host or a docker:dind container.

[0x2b3bfa0]

What kind of runner / What can be done	jobs.<job_id>.container, jobs.<job_id>.services and jobs.<job_id>.steps.*.uses with container actions	docker build & docker run
GitHub-hosted runner	✅	✅
self–hosted runner and cml runner 1	✅	✅
containerized self–hosted runner 2 with /var/run/docker.sock from host or docker:dind	🚫	✅
containerized self–hosted runner 3 with no access to /var/run/docker.sock	🚫	🚫

Footnotes

1. cml runner --repo=https://github.com/user/repository --token=ghp_... ↩

2. docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock iterativeai/cml runner --repo=https://github.com/user/repository --token=ghp_... ↩

3. docker run --rm iterativeai/cml runner --repo=https://github.com/user/repository --token=ghp_... ↩

[DavidGOrtega]

I was opening another issue to follow up discord/cml#951240652035883008

The idea was to throw an error explaining that volumes in GH are explicitly specified in the yaml instead of current one
Makes sense to follow it up here?

[0x2b3bfa0]

This issue is about containerized self–hosted runners. It doesn't seem to be related to the Discord question you link. 🤔

[dacbd]

we should try to document this somewhere more permanent than in an issue. perhaps the readme