Skip to content

DeimosCloud/terraform-google-gitlab-ci-runner

Repository files navigation

GCP GitLab Runner

This Terraform modules creates a GitLab CI runner.

The runners created by the module use preemptible instances by default for running the builds using the docker+machine executor.

  • Shared cache in GCS with life cycle management to clear objects after x days.
  • Runner agents registered automatically.

The runner supports:

GitLab CI docker-machine runner

In this scenario the runner agent is running on a GCP Compute Instance and runners are created by docker machine using preemptible instances. Runners will scale automatically based on the configuration. The module creates a GCS cache by default, which is shared across runners (preemptible instances).

GitLab CI docker runner

In this scenario not docker machine is used but docker to schedule the builds. Builds will run on the same compute instance as the agent.

GitLab CI Kubernetes Runner

See gke-runner

Autoscaling the Runners

Both docker-machine runner and docker runners autoscale using GCP Custom metrics. The runner publishes running jobs metrics to stackdriver which is then used to scale up/down the number of active runners. var.runners_min_replicas and var.runners_max_replicas defined variables for the minimum and maximum number of runners respectively. It uses Google Managed Instance Group Autoscaler to scale when the average of running jobs exceeds var.runners_concurrent.

⚠️ With scaling up and down, runners can get terminated without waiting for clean up when GCP scales down. GCP shutdown scripts have a maximum of atmost 90 seconds, during which it's forcefully terminated. Hence, it might not wait for your jobs to finish during scale down. This can leave orphan docker-machine-created instances and failed jobs. Use this with caution. Advisably, if you need more than one runner, set var.runner_min_replicas = var.runners_max_replicas = number of runners you need.

GitLab runner token configuration

The runner is registered on initial deployment. Each new runner registers itself with the same description and tag. To register the runner automatically set the variable gitlab_runner_registration_config["registration_token"]. This token value can be found in your GitLab project, group, or global settings. For a generic runner you can find the token in the admin section. By default the runner will be locked to the target project, not run untagged. Below is an example of the configuration map.

gitlab_runner_registration_config = {
  registration_token = "Required: <registration token>"
  tag_list           = "Optional: <your tags, comma separated>"
  description        = "Optional: <some description>"
  locked_to_project  = "optional: true"
  run_untagged       = "Optional: false"
  maximum_timeout    = "Optional: 3600"
  access_level       = "Optional: <not_protected OR ref_protected, ref_protected runner will only run on pipelines triggered on protected branches. Defaults to not_protected>"
}

GitLab runner cache

By default the module creates a a cache for the runner in Google Cloud Storage. Old objects are automatically removed via a configurable life cycle policy on the bucket.

Usage

module "runner" {
  source  = "DeimosCloud/gitlab-ci-runner/google"

  network    = "default"
  subnetwork = "someNetwork"
  region     = "europe-west1"
  project    = local.project_id

  runners_name       = "docker-default"
  runners_gitlab_url = "https://gitlab.com"

  gitlab_runner_registration_config = {
    registration_token = "my-token"
    tag_list           = "docker"
  }

}

NOTE: If runners are set to use internal IPs, a Cloud NAT must be deployed for runners to be able to reach internet

Contributing

Report issues/questions/feature requests on in the issues section.

Full contributing guidelines are covered here.

Requirements

Name Version
terraform >= 0.14

Providers

Name Version
google 4.19.0
random 3.1.3

Modules

Name Source Version
cache_gcs ./modules/cache n/a

Resources

Name Type
google_compute_firewall.docker_machine resource
google_compute_firewall.docker_machine_ssh resource
google_compute_firewall.docker_machines resource
google_compute_firewall.internet resource
google_compute_firewall.ssh resource
google_compute_instance_template.this resource
google_compute_region_autoscaler.this resource
google_compute_region_instance_group_manager.this resource
google_monitoring_metric_descriptor.jobs resource
google_project_iam_member.this resource
google_service_account.agent resource
google_service_account.runner resource
google_service_account_iam_member.agent_runner resource
google_service_account_key.agent resource
random_id.this resource
random_shuffle.zones resource
google_compute_network.this data source
google_compute_zones.available data source

Inputs

Name Description Type Default Required
cache_bucket_versioning Boolean used to enable versioning on the cache bucket, false by default. bool false no
cache_expiration_days Number of days before cache objects expires. number 2 no
cache_location The location where to create the cache bucket in. If not specified, it defaults to the region any null no
cache_shared Enables cache sharing between runners. bool true no
cache_storage_class The cache storage class string "STANDARD" no
create_cache_bucket Creates a cache cloud storage bucket if true bool true no
docker_machine_disk_size The disk size for the docker-machine instances. number 20 no
docker_machine_disk_type The disk Type for docker-machine instances. string "pd-standard" no
docker_machine_download_url Full url pointing to a linux x64 distribution of docker machine. string "https://gitlab-docker-machine-downloads.s3.amazonaws.com/main/docker-machine-Linux-x86_64" no
docker_machine_image A GCP custom image to use for spinning up docker-machines string "" no
docker_machine_machine_type The Machine Type for the docker-machine instances. string "f1-micro" no
docker_machine_options List of additional options for the docker machine config. Each element of this list must be a key=value pair. E.g. '["google-zone=a"]' list(string) [] no
docker_machine_preemptible If true, docker-machine instances will be premptible bool false no
docker_machine_tags Additional Network tags to be attached to the docker-machine instances. list(string) [] no
docker_machine_use_internal_ip If true, docker-machine instances will have only internal IPs. bool false no
gitlab_runner_registration_config Configuration used to register the runner. Available at https://docs.gitlab.com/ee/api/runners.html#register-a-new-runner. map
{
"access_level": "not_protected",
"description": "",
"locked_to_project": "",
"maximum_timeout": "",
"registration_token": "",
"run_untagged": "",
"tag_list": ""
}
no
gitlab_runner_version Version of the GitLab runner. Defaults to latest string "" no
labels Map of labels that will be added to created resources map(string) {} no
network The target VPC for the docker-machine and runner instances. string "default" no
prefix The prefix to apply to all GCP resource names (e.g. -runner, -agent-1). string "ci" no
project The GCP project to deploy the runner into. string n/a yes
region The GCP region to deploy the runner into. string n/a yes
runner_additional_service_account_roles Additional roles to pass to the Runner service account list(string) [] no
runners_additional_volumes Additional volumes that will be used in the runner config.toml, e.g Docker socket list(any) [] no
runners_allow_ssh_access Enables SSH Access to the runner instances. bool true no
runners_concurrent Concurrent value for the runners, will be used in the runner config.toml. Limits how many jobs globally can be run concurrently when running docker-machine. number 10 no
runners_disable_cache Runners will not use local cache, will be used in the runner config.toml bool false no
runners_disk_size The size of the created gitlab runner instances in GB. number 20 no
runners_disk_type The Disk type of the gitlab runner instances string "pd-standard" no
runners_docker_runtime docker runtime for runners, will be used in the runner config.toml string "" no
runners_enable_monitoring Installs Stackdriver monitoring Agent on runner Instances to collect metrics. bool true no
runners_environment_vars Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml list(string) [] no
runners_executor The executor to use. Currently supports docker+machine or docker. string "docker+machine" no
runners_gitlab_url URL of the GitLab instance to connect to. string "https://gitlab.com" no
runners_helper_image Overrides the default helper image used to clone repos and upload artifacts, will be used in the runner config.toml string "" no
runners_idle_count (docker-machine) Idle count of the runners, will be used in the runner config.toml. number 0 no
runners_idle_time (docker-machine) Idle time of the runners, will be used in the runner config.toml. number 600 no
runners_image Image to run builds, will be used in the runner config.toml string "docker:19.03" no
runners_install_docker_credential_gcr Install docker_credential_gcr inside startup_script_pre_install script bool true no
runners_limit Limit for the runners, will be used in the runner config.toml. number 0 no
runners_machine_autoscaling (docker-machine) Set autoscaling parameters based on periods, see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersmachine-section
list(object({
periods = list(string)
idle_count = number
idle_time = number
timezone = string
}))
[] no
runners_machine_type Instance type used for the GitLab runner. string "n1-standard-1" no
runners_max_builds (docker-machine) Max builds for each runner after which it will be removed, will be used in the runner config.toml. By default set to 0, no maxBuilds will be set in the configuration. number 0 no
runners_max_growth_rate (docker-machine) The maximum number of machines that can be added to the runner in parallel. Default is 0 (no limit). number 0 no
runners_max_replicas The maximum number of runners to spin up.For docker+machine, this is the max number of instances that will run docker-machine. For docker, this is the maximum number of runner instances. number 1 no
runners_metadata (Optional) Metadata key/value pairs to make available from within instances created from this template. map {} no
runners_min_replicas The minimum number of runners to spin up. For docker+machine, this is the min number of instances that will run docker-machine. For docker, this is the minimum number of runner instances number 1 no
runners_name Name of the runner, will be used in the runner config.toml. string n/a yes
runners_output_limit Sets the maximum build log size in kilobytes, by default set to 4096 (4MB) number 4096 no
runners_post_build_script Commands to be executed on the Runner just after executing the build, but before executing after_script. string "\"\"" no
runners_pre_build_script Script to execute in the pipeline just before the build, will be used in the runner config.toml string "\"\"" no
runners_pre_clone_script Commands to be executed on the Runner before cloning the Git repository. this can be used to adjust the Git client configuration first, for example. string "\"\"" no
runners_preemptible If true, runner compute instances will be premptible bool true no
runners_privileged Runners will run in privileged mode, will be used in the runner config.toml bool true no
runners_pull_policy pull_policy for the runners, will be used in the runner config.toml string "always" no
runners_request_concurrency Limit number of concurrent requests for new jobs from GitLab (default 1) number 1 no
runners_root_size Runner instance root size in GB. number 16 no
runners_services_volumes_tmpfs n/a
list(object({
volume = string
options = string
}))
[] no
runners_shm_size shm_size for the runners, will be used in the runner config.toml number 0 no
runners_ssh_allowed_cidr_blocks List of CIDR blocks to allow SSH Access to the gitlab runner instance. list(string)
[
"0.0.0.0/0"
]
no
runners_tags Additional Network tags to be attached to the Gitlab Runner. list(string) [] no
runners_target_autoscale_cpu_utilization The target CPU utilization that the autoscaler should maintain. If runner CPU utilization gets above this, a new runner is created until runners_max_replicas is reached number 0.9 no
runners_use_internal_ip Restrict runners to the use of a Internal IP address. NOTE: NAT Gateway must be deployed in your network so that Runners can access resources on the internet bool false no
runners_volumes_tmpfs n/a
list(object({
volume = string
options = string
}))
[] no
startup_script_post_install Startup script snippet to insert after GitLab runner install string "" no
startup_script_pre_install Startup script snippet to insert before GitLab runner install string "" no
subnetwork Subnetwork used for hosting the gitlab-runners. string "" no

Outputs

No outputs.