Staves

Staves is a container image builder based on the Portage package manager for Gentoo Linux. Staves leverages Gentoo's infrastructure to build highly customized application images optimized for image size, speed, and security.

Staves is in alpha status and is not recommended for production use

Features

• Minimal dependencies result in small image sizes and reduced attack surface
• Feature toggles provide even more fine-grained control over image size and attack surface
• Full control over the build process (e.g. compiler settings allows for images customized to a specific Platform or CPU
• Access to hardened build toolchains with things like PIC (Position Independent Code) and SSP (Stack Smashing Protection)

Table of Contents

• Installation
• Getting Started
  • Customizing the Image
  • Package-specific configuration
  • Build environment customization
  • Custom package repositories
• How it works
• How to build images based on musl libc
• Comparison to other tools
  • Docker multistage builds
  • Buildkit, buildctl, img, buildx
  • Kaniko, makisu
  • buildah
  • Bazel

Installation

Staves is not available via PyPI at the moment. The following instructions will guide you through the installation from source.

Make sure you have Docker and the Poetry package manager installed. Then clone the repository and use poetry to set up the project:

$ git clone https://github.com/digitalernachschub/staves.git
$ cd staves
$ poetry install

Getting started

Staves images are defined declaratively using TOML markup. Store the following image specification in a file called staves.toml:

name = 'staves/bash'
packages = ['app-shells/bash']
command = ['/bin/bash', '-c']

This file tells Staves to bundle the Gentoo package "app-shells/bash" and its runtime dependencies into a container image tagged "staves/bash". The image's entrypoint is defined as "/bin/bash -c". Most likely, you are not running Gentoo as your host system. Therefore, Staves relies on a builder image. We can use any stage3 for that purpose, so let's pull one of the official Docker images:

$ docker pull gentoo/stage3-amd64-hardened-nomultilib

Now we are ready to start the build:

$ poetry run staves build --builder gentoo/stage3-amd64-hardened-nomultilib --build-cache staves

This will take some time. The command performs the following steps:

• Download the official Docker image of the latest package list (i.e. Portage snapshot)
• Create a binary package from every package in the builder. These binary packages are cached in the Docker volume staves to speed up subsequent builds.
• Install app-shells/bash and its runtime dependencies into /tmp/rootfs of the build container
• Create the file staves_root.tar in your working directory from the contents of /tmp/rootfs
• Create and tag a Docker image with the contents of the tarball

Once the command has finished, we can test the newly created image:

$ docker run --rm staves/bash "echo Hello World!"

Customizing the image

The previous example uses only the most basic staves.toml, but there are several ways to customize the resulting image.

Package-specific configuration

A Staves image specification can control individual feature toggles of a package. In Gentoo Linux, these are called USE flags. The app-shells/bash package, for example, can also be compiled without Native Language Support. We do this by specifying a TOML table with the package name as the identifier and the USE flags as an array:

['app-shells/bash']
use = ['-nls']

For one, this will shave off a couple of megabytes from the resulting image: When using a musl libc toolchain, the resulting image is 9.29 MB which is 29% smaller than the official Bash image (13.1 MB). For another, there are cases where disabling functionality will reduce the attack surface of the resulting image. See packages.gentoo.org for installable packages and their USE flags.

Build environment customization

Staves allows adjusting the global build environment. For example, we can enable aggressive compiler optimizations via CFLAGS or enable support for the AVX instruction set:

[env]
CFLAGS="${CFLAGS} -O3"
CPU_FLAGS_X86="${CPU_FLAGS_X86} avx"

Values in the env section of a staves.toml will be appended to the make.conf of the builder and are applied to all packages. See the make.conf.example file for a documentation of allowed values.

However, it is also possible to apply package-specific configurations to the build environment. The env table can have custom attributes that represent environment configurations themselves. These environments can be applied to individual packages using the env attribute of a package configuration:

[env.nocache]
FEATURES="-buildpkg"

['=dev-utils/mylibrary-9999']
env = ['nocache']

Technically, the env.nocache section will create the file /etc/portage/env/nocache. This environment is then applied to the specified package using a corresponding entry in /etc/portage/package.env.

Custom package repositories

Staves supports custom repositories to be set up in the builder. For each repository, add the following section to your staves.toml:

[[repositories]]
name = 'my-repo'
type = 'git'
uri = 'ssh://[email protected]/my-overlay.git'

This will create a corresponding file in /etc/portage/repos.d/my-repo and run emaint sync --repo my-repo to fetch the most recent package list. Note that dev-vcs/git is not included in an official Stage 3 tarball. It is your responsibility ensure that your builder image contains the dependencies necessary to fetch the repository.

How it works

Staves consists of two parts, a host part and a builder part. The host part provides the command-line interface and parses the staves.toml file. The builder part controls the process inside the build container.

When a user invokes the build command, Staves pulls the newest Docker image containing a Portage snapshot and creates a container from it. Staves creates a build container from the provided stage3 builder image where the Portage tree is mounted into. Staves copies the builder part into the container and runs it. The builder script reads the build configuration (USE flags, make.conf, …) from stdin, applies the changes to the build environment and runs a couple of emerge commands to generate the root filesystem of the application image. Once the builder has finished, the host part extracts a tar archive from the container and assembles a Docker image from it.

How to build images based on MUSL libc

Building images based on anything else than GLibc will require you to prepare a stage3 image with a corresponding toolchain. The official Gentoo docker images do not include a stage3 with a MUSL toolchain at the time of writing (2020-10-23), but there are several other ways to achieve this. For example, you can use Catalyst or GRS to bootstrap the corresponding system. This how-to will use the Docker image generator gentoo-docker-images to create a MUSL stage3 for amd64.

As a prerequisite, you need to have the buildx Docker extension installed. If not, the following instructions will build and install the buildx command:

$ export DOCKER_BUILDKIT=1
$ docker build --platform=local -o . git://github.com/docker/buildx
$ mkdir -p ~/.docker/cli-plugins
$ mv buildx ~/.docker/cli-plugins/docker-buildx

Now we can clone the gentoo-docker-images repository and run build.sh with the appropriate TARGET environment.

$ git clone https://github.com/gentoo/gentoo-docker-images.git
$ cd gentoo-docker-images
$ TARGET=stage3-amd64-musl-hardened ./build.sh

This will create a docker image tagged as gentoo/stage3:amd64-musl-hardened. Since Staves works with any stage3, this image can simply be used as a builder to produce MUSL-based images.

Comparison to other tools

OCI images are built around the idea of different layers that are merged together to create a container filesystem. This approach promotes reusability of layers and permits layer caching. The drawback is that the number of layers required for a single image adds up quickly. Since these layers typically come from different entities and organizations, you have to trust each of them to provide uncompromised images. If any of the layers are compromised, your final image will likely be compromised, too.

Staves does not rely on layering to provide the final image. Every image is built from scratch. You need to trust the Gentoo builder image and the packages you are installing. From there, everything is built from the sources. The security of this process is arguably easier to verify than the layering approach involving many different parties.

Docker multistage builds

Multistage builds operate on the same principle as Staves in that both separate build-time and run-time dependencies. In multistage builds, the installation steps need to be invoked manually, but you are free to choose any build image or package manager to perform the steps. This results in an imperative image definition with several RUN steps. For multi-platform builds, these need to be parameterized with build arguments.

Staves is limited to Gentoo builders, but provides a declarative way to create images. The prerequisite is that there is a Gentoo ebuild for the installed packages. There is no need to parameterize for multiple platforms, because platform specific steps are performed in the ebuilds.

buildkit, buildctl, img, buildx

Buildkit improves upon regular docker build invocations, because it can be executed without root privileges, has a pluggable frontend (i.e. "image format"), and a number of other goodies. Buildkit uses a library-first approach and is used by different command-line tools, such as buildctl, img and buildx.

Staves currently depends on Docker and defines a custom image format. Buildkit and its descendants compare similarly to Staves as as Docker multistage builds. They are more flexible, becase you can depend on other base images and choose any suitable builder as an image. Staves makes use of Gentoo's Portage package manager and therefore it relies on Gentoo infrastructure and depends on a Gentoo stage3 builder image.