Skip to content

Commit

Permalink
Fix formatting issues in YAML and Bash files
Browse files Browse the repository at this point in the history
  • Loading branch information
vbem committed Feb 22, 2024
1 parent 0296e6f commit b0e37b1
Show file tree
Hide file tree
Showing 6 changed files with 100 additions and 67 deletions.
1 change: 0 additions & 1 deletion .github/workflows/linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,3 @@ jobs:
packages: read
statuses: write
uses: vbem/reusable/.github/workflows/superlinter.yml@v1
...
3 changes: 1 addition & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:

jobs:
test:
runs-on: [self-hosted, '${{ github.repository }}']
runs-on: [self-hosted, "${{ github.repository }}"]

steps:
- run: |
Expand All @@ -19,4 +19,3 @@ jobs:
curl myip.ipip.net
- run: |
curl -s -4 icanhazip.com
...
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# multi-runners

[![awesome-runners](https://img.shields.io/badge/listed%20on-awesome--runners-blue.svg)](https://github.com/jonico/awesome-runners)
[![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/vbem/multi-runners?label=Release&logo=github)](https://github.com/vbem/multi-runners/releases)
[![Linter](https://github.com/vbem/multi-runners/actions/workflows/linter.yml/badge.svg)](https://github.com/vbem/multi-runners/actions/workflows/linter.yml)
Expand All @@ -7,14 +8,17 @@
🌈🌈🌈 **Multi self-hosted GitHub action runners on single host!** 🌈🌈🌈

## Introduction

This application is designed for controlling multi [self-hosted GitHub Action runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners) on single host, when [Actions Runner Controller (ARC)](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller) is not feasible in your engineering environment. This application has following advantages:

- Only single Linux host required.
- Simple as more as possible, single Bash script.
- Lightweight wrapper of official self-hosted runner.
- Both *github.com* and *GitHub Enterprise* are support.
- Both *organization* and *repository* level runners are supported.

## Usage

```plain
mr.bash - https://github.com/vbem/multi-runners
Expand Down Expand Up @@ -49,6 +53,7 @@ Options:
```

### Download this application

This application requires to be run under a Linux user with **non-password sudo permission** (`%runners ALL=(ALL) NOPASSWD:ALL`). It's also fine to run this application by `root`:

```bash
Expand All @@ -58,6 +63,7 @@ cd multi-runners
```

### Setup PAT

This application requires a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with smallest permissions and shortest expiration time. Only `add`/`del`/`pat2token` sub-commands need this PAT. You can remove it on *GitHub* after multi-runners' setup.

PAT types | Repository level runners | Organization level runners
Expand All @@ -74,11 +80,13 @@ ALL_PROXY=socks5h://localhost
```

You can run following command to check whether or not your PAT can generate [GitHub Actions runners' registration-token](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners#authentication-requirements):

```bash
./mr.bash pat2token --org <ORG-NAME> --repo <REPO-NAME>
```

### Download the latest version of GitHub Actions package

If environment variable `MR_RELEASE_URL` is empty, this application will download the [latest version of GitHub Actions runners tar package](https://github.com/actions/runner/releases) to local directory `/tmp/` during runtime.

```bash
Expand All @@ -88,10 +96,13 @@ If environment variable `MR_RELEASE_URL` is empty, this application will downloa
If limited by slow download speed, you can also manually download it to `/tmp/`, and set the `MR_RELEASE_URL` env as `/tmp/actions-runner-linux-x64-2.345.6.tar.gz`.

### GitHub Enterprise Server editions

*GitHub Enterprise Server* editions usually have different server and API URL prefixes comparing with *github.com*, you can set them in environment variables `MR_GIHUB_BASEURL` and `MR_GIHUB_API_BASEURL`.

### Setup multi-runners on single host

To setup multi-runners, you can simplify run following command multi times:

```bash
# 1 runner for repository `<ORG-NAME-1>/<REPO-NAME-1>`
./mr.bash add --org <ORG-NAME-1> --repo <REPO-NAME-1>
Expand All @@ -107,11 +118,15 @@ To setup multi-runners, you can simplify run following command multi times:
```

### List all runners on current host

This application also integrated status check of runners.

```bash
./mr.bash list
```

Which outputs,

```bash
runner-0 537M running https://github.com/<ORG-NAME-1>/<REPO-NAME-1>
runner-1 537M running https://github.com/<ORG-NAME-1>/<REPO-NAME-2>
Expand All @@ -122,13 +137,17 @@ runner-5 537M running https://github.com/<ORG-NAME-2>
```

### Delete an existing runner

You can delete an existing runner by its local Linux username.

```bash
./mr.bash del --user <runner-?>
```

### Specify runner in workflow file

In [`jobs.<job_id>.runs-on`](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idruns-on), target runners can be based on the labels as follows via [GitHub context](https://docs.github.com/en/actions/learn-github-actions/contexts#github-context):

```yaml
# For organization level self-hosted runners
runs-on: [self-hosted, '${{ github.repository_owner }}']
Expand All @@ -138,11 +157,15 @@ runs-on: [self-hosted, '${{ github.repository }}']
```
### Set environment variables into runners process
As described in GitHub official document, there's an approach to [inject environment variables into runners process](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-a-proxy-server-with-self-hosted-runners#using-a-env-file-to-set-the-proxy-configuration) via the `.env` file before configuring or starting the self-hosted runners. This can be achieved via the `--dotenv` option, for example:

```bash
./mr.bash add --org <ORG> --repo <REPO> --dotenv 'TZ=Asia/Shanghai' --dotenv 'PATH=\$PATH:/mybin'
```

Then the following lines will be added to `.env` file located in self-hosted runner's directory before its configuring and starting:

```plain
TZ=Asia/Shanghai
PATH=$PATH:/mybin
Expand All @@ -155,21 +178,26 @@ A multi-national corporation adopted GitHub as its centralized engineering effic
In such a bad situation, we still need to setup reliable self-hosted runners in this country. What should we do? 🤣

A cost-conscious solution can be described as following architecture:

```plain
Endpoints <-------- VM-Runners ----> Firewall ----> VM-Proxy ----> GitHub
\ / | \
-------------------------- | ----> Other endpoints
Branch office network Remote Proxy
```

A host *VM-Runners* is required for self-hosted runners, which is placed in this country and:

- Can access endpoints of this country branch
- Can NOT access *GitHub* directly or stably

A tiny specification host *VM-Proxy* is required as ***Remote Proxy***, which is deployed in a place that:

- Can access *GitHub* directly and stably
- Can be accessed by *VM-Runners* directly and stably

Meanwhile, **outbound traffics from *VM-Runners* MUST be routed by predefined rules**:

- [Requests to *GitHub* endpoints](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#communication-requirements) and non-local endpoints should be forward to the *Remote Proxy* on *VM-Proxy*
- Requests to local endpoints should be handled directly

Expand All @@ -182,6 +210,7 @@ Before setup runners on *VM-Runners*, we need a [***Local Proxy***](https://docs
Usually firstly we need to setup the client of selected *Remote Proxy* which exposes a SOCKS5 proxy on *VM-Runners* (this local SOCKS5 localhost port will unconditionally forward all traffics to *VM-Proxy*), and then setup a [*privoxy*](https://www.privoxy.org/) on top of previous local SOCKS5 for [domain based forwarding](https://www.privoxy.org/user-manual/config.html#SOCKS). These configurations are complex and prone to errors. Via [*Clash*](https://github.com/Dreamacro/clash), we can combine both client of *Remote Proxy* and domain based forwarding into only one *Local Proxy*. The example configuration file and startup script of *Clash* and given in this repo's [clash.example/](clash.example/) directory.

Assume the *Local Proxy* was launched as `socks5h://localhost:7890`, we can test it via following commands on *VM-Runners*:

```bash
# Without *Local Proxy*, it will print outbound public IP of *VM-Runners*
curl -s -4 icanhazip.com
Expand All @@ -193,12 +222,14 @@ all_proxy=socks5h://localhost:7890 curl -s -4 icanhazip.com
When *Local Proxy* is ready, we start self-hosted runners' setup on *VM-Runners*.

As *VM-Runners* Can NOT access *GitHub* directly or stably, use *Local Proxy* to clone this repository:

```bash
all_proxy=socks5h://localhost:7890 git clone https://github.com/vbem/multi-runners
cd multi-runners
```

As self-hosted runners' tar package downloading and registration-token fetching also requires communication with GitHub, we also configure *Local Proxy* for this application:

```bash
cat > .env <<- __
MR_GITHUB_PAT='<paste-for-GitHub-PAT-here>'
Expand All @@ -207,27 +238,32 @@ __
```

To download the self-hosted runners' tar package from *GitHub*:

```bash
./mr.bash download
```

To validate your *PAT* has sufficient permissions for self-hosted runners registration on your GitHub organization `https://github.com/<ORG-NAME>`:

```bash
./mr.bash pat2token --org <ORG-NAME>
```

To setup two self-hosted runners on *VM-Runners* for your GitHub organization:

```bash
./mr.bash add --org <ORG-NAME> --dotenv 'all_proxy=socks5h://localhost:7890'
./mr.bash add --org <ORG-NAME> --dotenv 'all_proxy=socks5h://localhost:7890'
```

To check the status of self-hosted runners:

```bash
./mr.bash list
```

To check the *Local Proxy* works well in your runners' process, you can add a simple workflow `.github/workflows/test-local-proxy.yml` in your repository. If `icanhazip.com` was configured as a following-to-remote domain, the workflow run will print outbound public IP of *VM-Proxy*, even though this workflow actually runs on *VM-Proxy*.

```yaml
---
name: Test Local Proxy works in my self-hosted runners
Expand Down
3 changes: 1 addition & 2 deletions clash.example/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ proxies:
cipher: chacha20-ietf-poly1305 # 👈 TODO: paste your `ss-server` cipher
server: ss.example.com # 👈 TODO: paste your `ss-server` address
port: 12345 # 👈 TODO: paste your `ss-server` port
password: 'Berlin Wall has long been down' # 👈 TODO: paste your `ss-server` password
password: "Berlin Wall has long been down" # 👈 TODO: paste your `ss-server` password

# only route GitHub and specific traffics to remote proxy
mode: rule
Expand All @@ -31,4 +31,3 @@ rules:

# defaults to direct
- MATCH,DIRECT
...
14 changes: 7 additions & 7 deletions clash.example/start-clash.bash
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ declare -rg clash_mixed_port
declare -rg clash_mixed_port_publish_ip='127.0.0.1'

docker run \
--name "$(basename "$clash_image"|cut -d: -f1)" \
--detach \
--restart always \
--env TZ='Asia/Shanghai' \
--volume "$clash_config_path:/root/.config/clash/config.yaml:ro" \
--publish "$clash_mixed_port_publish_ip:$clash_mixed_port:$clash_mixed_port" \
"$clash_image"
--name "$(basename "$clash_image" | cut -d: -f1)" \
--detach \
--restart always \
--env TZ='Asia/Shanghai' \
--volume "$clash_config_path:/root/.config/clash/config.yaml:ro" \
--publish "$clash_mixed_port_publish_ip:$clash_mixed_port:$clash_mixed_port" \
"$clash_image"
110 changes: 55 additions & 55 deletions mr.bash
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,12 @@ function log::_ {
&& pos="/$each$pos"
done
case "$1" in
FATAL) color="5;1;91" ;;
ERR*) color="1;91" ;;
WARN*) color="95" ;;
INFO* | NOTICE) color="92" ;;
DEBUG) color="94" ;;
*) color="96" ;;
FATAL) color="5;1;91" ;;
ERR*) color="1;91" ;;
WARN*) color="95" ;;
INFO* | NOTICE) color="92" ;;
DEBUG) color="94" ;;
*) color="96" ;;
esac
datetime="\e[3;2;90m$(date -Isecond)\e[0m"
pos="\e[3;90m${pos:1}\e[0m"
Expand Down Expand Up @@ -354,62 +354,62 @@ function mr::main {

while true; do
case "$1" in
-h | --help) echo -n "$HELP" && return ;;
--org)
org="$2"
shift 2
;;
--repo)
repo="$2"
shift 2
;;
--user)
user="$2"
shift 2
;;
--labels)
labels="$2"
shift 2
;;
--token)
token="$2"
shift 2
;;
--group)
group="$2"
shift 2
;;
--dotenv)
dotenv+="$2"$'\n'
shift 2
;;
--)
shift
break
;;
*)
log::_ ERROR "Invalid option '$1'! See '$FILE_THIS help'."
return 255
;;
-h | --help) echo -n "$HELP" && return ;;
--org)
org="$2"
shift 2
;;
--repo)
repo="$2"
shift 2
;;
--user)
user="$2"
shift 2
;;
--labels)
labels="$2"
shift 2
;;
--token)
token="$2"
shift 2
;;
--group)
group="$2"
shift 2
;;
--dotenv)
dotenv+="$2"$'\n'
shift 2
;;
--)
shift
break
;;
*)
log::_ ERROR "Invalid option '$1'! See '$FILE_THIS help'."
return 255
;;
esac
done

# parse sub-commands into functions
subCmd="$1"
shift
case "$subCmd" in
add) mr::addRunner "$user" "$org" "$repo" "$token" "$labels" "$group" "$dotenv" ;;
del) mr::delRunner "$user" "$org" "$repo" "$token" ;;
list) mr::listRunners ;;
status) mr::statusRunner "$user" ;;
download) mr::downloadRunner ;;
pat2token) mr::pat2token "$org" "$repo" ;;
help | '') echo -n "$HELP" >&2 ;;
test) mr::test "$@" ;;
*)
log::_ ERROR "Invalid command '$1'! See '$FILE_THIS help'."
return 255
;;
add) mr::addRunner "$user" "$org" "$repo" "$token" "$labels" "$group" "$dotenv" ;;
del) mr::delRunner "$user" "$org" "$repo" "$token" ;;
list) mr::listRunners ;;
status) mr::statusRunner "$user" ;;
download) mr::downloadRunner ;;
pat2token) mr::pat2token "$org" "$repo" ;;
help | '') echo -n "$HELP" >&2 ;;
test) mr::test "$@" ;;
*)
log::_ ERROR "Invalid command '$1'! See '$FILE_THIS help'."
return 255
;;
esac
}

Expand Down

0 comments on commit b0e37b1

Please sign in to comment.