⚠️ These notes are still a work in progress.

bazel-workshop

Prerequisites

• Install Bazel: bazel.build/install
  • Recommended method via Bazelisk:
    • cd /usr/sbin
    • wget --progress=dot:giga https://github.com/bazelbuild/bazelisk/releases/download/v1.10.0/bazelisk-linux-amd64
    • chmod +x bazelisk-linux-amd64
    • ln -s /usr/sbin/bazelisk-linux-amd64 /usr/sbin/bazel
    • Use a .bazelversion file to set the Bazel version for each repository.
  • Verify that all participants use the same version!
  • Alternatively: use a build environment, e.g. Docker container
• (Optional) Clone examples repository: github.com/bazelbuild/examples

What do you want to learn in this workshop?

• What do you already know about Bazel?
• More theory or more hands-on?
• Which advanced topics are you interested in? E.g.:
  • distributed builds (remote caching, remote execution)
  • query language
  • build file generator (Gazelle)
  • bazel-diff
  • custom Bazel rules (with unit tests)

Resources to find general information

• Project website: bazel.build
• Official Bazel reference
• bazel help command
• Bazel glossary
• FAQ
• Official Bazel slack: slack.bazel.build
• Source code github.com/bazelbuild

What is Bazel?

Bazel is an open-source build system that is intended for multi-language mono-repos. Bazel is focused on fast, scalable, parallel and reproducible builds. A core strength of Bazel is an extensive caching framework, based on a strict and fine grained build graph.

Some benefits of Bazel:

• multi-language, multi platform, extensible system
  • ideal for multilingual mono-repos
  • can support multiple architectures, configurations etc.
• free open-source project, with active community
• clear file and target structure
• reproducible builds
• hermetic (sand-boxed) execution with strictly defined in- and outputs
• build profiling and metrics
  • easy to get information, e.g. about performance bottle necks for further optimization
• powerful query language (full query guide)
  • easy to analyze service architecture, create dependency graphs etc.
• granular targets and dependencies, no circular dependencies allowed, strict visibility control
  • helps to avoid regression and keep services separated
  • know which unit tests need to run for any change
  • only build and test what is really needed
• incremental builds
  • faster iteration and development
• (remote) caching
  • faster builds and CI
• parallelization and scalability, remote execution, distributed builds

See also the intro and Bazel vision pages.

Possible limitations:

• steep learning curve for maintainers
• running services and integration tests, that need extended system access, goes against the philosophy of sand-boxing
• languages that are (yet) without rule sets

Bazel basics

Starlark

The Starlark configuration language is a deterministic, hermetic and parallelizable Python dialect.

Buildifier Starlark formatter

To ensure the standardized formatting of all BUILD.bazel and .bzl files the Starlark formatter buildifier can be used.

File structure

The basic structure of the Bazel files in a repository could, for example, look like this:

.
├── WORKSPACE.bazel
├── .bazelrc
├── BUILD.bazel
├── bazel
│   ├── bazelrcs
│   │   ├── vm.bazelrc
│   │   └── ...
│   ├── external
│   │   ├── BUILD.bazel
│   │   ├── sentry_native.BUILD
│   │   ├── requirements.in
│   │   ├── requirements.txt
│   │   └── ...
│   ├── scripts
│   │   ├── bazel_diff.sh
│   │   ├── run_buildifier.sh
│   │   └── ...
│   ├── BUILD.bazel
│   ├── cpp_repositories.bzl
│   ├── go_repositories.bzl
│   ├── python_repositories.bzl
│   ├── python_swagger.bzl
│   └── ...
├── lte
│   ├── gateway
│   │   └── python
│   │       ├── magma
│   │       │   ├── mobilityd
│   │       │   │   ├── tests
│   │       │   │   │   └── BUILD.bazel
│   │       │   │   └── BUILD.bazel
│   │       │   └── policydb
│   │       │       ├── servicers
│   │       │       │   └── BUILD.bazel
│   │       │       ├── tests
│   │       │       │   └── BUILD.bazel
│   │       │       └── BUILD.bazel
│   │       └── BUILD.bazel
│   ├── protos
│   │   ├── oai
│   │   │   └── BUILD.bazel
│   │   └── BUILD.bazel
│   └── swagger
│       └── BUILD.bazel
.

• WORKSPACE.bazel: Defines a Bazel project. Configuration of Bazel rules to be used in the project.
• .bazelrc: General configuration of flags for commands.
• BUILD.bazel: A file, e.g. lte/gateway/python/magma/mobilityd/BUILD.bazel, in which sources are defined in targets, e.g. binaries, libraries and tests. A BUILD.bazel file defines a Bazel package.
• bazel/: Centralized folder for custom Bazel definitions and configuration of external sources. This folder could also be called something like tools, this depends on the project. The same is true for all sub-folders mentioned here.
• bazel/scripts/: Centralized folder for Bash wrapper scripts for Bazel, e.g. for running the Starlark formatter or Bazel-diff.
• bazel/bazelrcs/name.bazelrc: Configuration of flags for commands in different environments.
• bazel/name.bzl
  • Custom Bazel code - these are custom Bazel rules, configuration of the tool chain and external dependencies.
  • In this example, cpp, go and python_repositories.bzl, specify external repositories. These can be e.g. local repositories (folders in the environment) or git repositories.
• bazel/external/name.BUILD: .BUILD files for external repositories that are not bazelified.

Build files

• General info
• Build file style guide

Targets and rules

The content of the build files are Targets. There are three basic types of rules: *_library, *_binary and *_test.

General infos on rules can be found in the Bazel docs.

Supported languages and formats are imported from:

• Generic rules (genrule)
• List of some rule repositories, this list is not comprehensive
• C/C++
• Go
• Python
• Java
  • jvm
• Javascript/Typescript:
  • rules_ts
  • nodejs (unmaintained)
  • rules_typescript, see also this subfolder
  • Typescript (moved to the above link)
• Bash
• Docker
• Rust
• Apple platform
• Haskell
• Packages (deb, rpm, zip, ...)
• ... and many more.

Commands

The Bazel commands are structured around verbs like build, test or query.

See also:

• complete list of available commands.
• build docs
• Specifying targets on the command line
• user manual
• command line reference

The bazel help and bazel info commands can also be helpful.

Build phases

• Loading phase
• Analysis phase
• Execution phase

Caching

Bazel has a very extensive and granular system of caching intermediate build outputs, artifacts and even test results. Builds that have many cache hits are much faster than builds without cache hits. The number of cache hits is reported at the end of each build, e.g. as INFO: 4449 processes: 703 disk cache hit, 2377 internal, 1367 processwrapper-sandbox, 2 worker..

Whenever cache-able results are produced, a local cache entry is generated in the disk cache. Some of the cache locations can be found by running bazel info and looking e.g. at the repository_cache entry.

Dependencies

• External dependencies

Inputs and outputs

A note on build environments and containers...

• Sand-boxing

• Bazel output directory

• The bazel clean command

Advanced topics

Querying

• Dependency graph and loading phase
  • bazel query
• Action graph
  • aquery
• cquery and analysis phase

Tinder/bazel-diff

• Target diff tool
• Only build/test what is really needed
• Use a wrapper script, e.g. like this.

Remote caching

• See https://bazel.build/docs/remote-caching
  • Use e.g. Google Cloud Platform
• buchgr/bazel-remote
• Managed remote caching services, e.g. buildbuddy

CI with Bazel

• Workflow setup
• Combining advanced topics like remote caching, bazel-diff, ...

Bazel Gazelle

• Build file generator
• Natively supports Go and Protobuf

Remote execution

• (difficult: is there a free service to try it?)

Custom rules

Writing your own rules/extending Bazel

• See e.g.
  • https://bazel.build/rules/rules-tutorial
  • https://bazel.build/extending/rules
• Danger zone!
  • This is where most bugs are coming from
  • Careful to keep it efficient and safe

Starlark tests

• https://github.com/bazelbuild/rules_testing