|
| 1 | +# MetaProtocolProxy CI |
| 2 | + |
| 3 | +MetaProtocolProxy uses the Envoybuilder image for CI. |
| 4 | + |
| 5 | +ci_api.sh is used to build the API proto files. |
| 6 | +ci_dev.sh is used to build the MetaProtocolProxy binary. |
| 7 | + |
| 8 | +# Developer use of CI Docker images |
| 9 | + |
| 10 | +There are two available flavors of Envoy Docker images for Linux, based on Ubuntu and Alpine Linux |
| 11 | +and an image based on Windows2019. |
| 12 | + |
| 13 | +## Ubuntu Envoy image |
| 14 | + |
| 15 | +The Ubuntu based Envoy Docker image at [`envoyproxy/envoy-build:<hash>`](https://hub.docker.com/r/envoyproxy/envoy-build/) is used for CI checks, |
| 16 | +where `<hash>` is specified in [`envoy_build_sha.sh`](https://github.com/envoyproxy/envoy/blob/main/ci/envoy_build_sha.sh). Developers |
| 17 | +may work with the latest build image SHA in [envoy-build-tools](https://github.com/envoyproxy/envoy-build-tools/blob/main/toolchains/rbe_toolchains_config.bzl#L8) |
| 18 | +repo to provide a self-contained environment for building Envoy binaries and running tests that reflects the latest built Ubuntu Envoy image. |
| 19 | +Moreover, the Docker image at [`envoyproxy/envoy:dev-<hash>`](https://hub.docker.com/r/envoyproxy/envoy/tags?name=dev) is an image that has an Envoy binary at `/usr/local/bin/envoy`. |
| 20 | +The `<hash>` corresponds to the main commit at which the binary was compiled. Lastly, `envoyproxy/envoy:dev` contains an Envoy |
| 21 | +binary built from the latest tip of main that passed tests. |
| 22 | + |
| 23 | +## Alpine Envoy image |
| 24 | + |
| 25 | +Minimal images based on Alpine Linux allow for quicker deployment of Envoy. The Alpine base image is only built with symbols stripped. |
| 26 | +To get the binary with symbols, use the corresponding Ubuntu based debug image. The image is pushed with two different tags: |
| 27 | +`<hash>` and `latest`. Parallel to the Ubuntu images above, `<hash>` corresponds to the |
| 28 | +main commit at which the binary was compiled, and `latest` corresponds to a binary built from the latest tip of main that passed tests. |
| 29 | + |
| 30 | +## Windows 2019 Envoy image |
| 31 | + |
| 32 | +The Windows 2019 based Envoy Docker image at [`envoyproxy/envoy-build-windows2019:<hash>`](https://hub.docker.com/r/envoyproxy/envoy-build-windows2019/) |
| 33 | +is used for CI checks, where `<hash>` is specified in [`envoy_build_sha.sh`](https://github.com/envoyproxy/envoy/blob/main/ci/envoy_build_sha.sh). |
| 34 | +Developers may work with the most recent `envoyproxy/envoy-build-windows2019` image to provide a self-contained environment for building Envoy binaries and |
| 35 | +running tests that reflects the latest built Windows 2019 Envoy image. |
| 36 | + |
| 37 | +# Build image base and compiler versions |
| 38 | + |
| 39 | +Currently there are three build images for Linux and one for Windows: |
| 40 | + |
| 41 | +* `envoyproxy/envoy-build` — alias to `envoyproxy/envoy-build-ubuntu`. |
| 42 | +* `envoyproxy/envoy-build-ubuntu` — based on Ubuntu 20.04 (Focal) with GCC 9 and Clang 14 compiler. |
| 43 | +* `envoyproxy/envoy-build-centos` — based on CentOS 7 with GCC 9 and Clang 14 compiler, this image is experimental and not well tested. |
| 44 | +* `envoyproxy/envoy-build-windows2019` — based on Windows ltsc2019 with VS 2019 Build Tools, as well as LLVM. |
| 45 | + |
| 46 | +The source for these images is located in the [envoyproxy/envoy-build-tools](https://github.com/envoyproxy/envoy-build-tools) |
| 47 | +repository. |
| 48 | + |
| 49 | +We use the Clang compiler for all Linux CI runs with tests. We have an additional Linux CI run with GCC which builds binary only. |
| 50 | + |
| 51 | +# C++ standard library |
| 52 | + |
| 53 | +As of November 2019 after [#8859](https://github.com/envoyproxy/envoy/pull/8859) the official released binary is |
| 54 | +[linked against libc++ on Linux](https://github.com/envoyproxy/envoy/blob/main/bazel/README.md#linking-against-libc-on-linux). |
| 55 | +To override the C++ standard library in your build, set environment variable `ENVOY_STDLIB` to `libstdc++` or `libc++` and |
| 56 | +run `./ci/do_ci.sh` as described below. |
| 57 | + |
| 58 | +# Building and running tests as a developer |
| 59 | + |
| 60 | +The `./ci/run_envoy_docker.sh` script can be used to set up a Docker container on Linux and Windows |
| 61 | +to build an Envoy static binary and run tests. |
| 62 | + |
| 63 | +The build image defaults to `envoyproxy/envoy-build-ubuntu` on Linux and |
| 64 | +`envoyproxy/envoy-build-windows2019` on Windows, but you can choose build image by setting |
| 65 | +`IMAGE_NAME` in the environment. |
| 66 | + |
| 67 | +In case your setup is behind a proxy, set `http_proxy` and `https_proxy` to the proxy servers before |
| 68 | +invoking the build. |
| 69 | + |
| 70 | +```bash |
| 71 | +IMAGE_NAME=envoyproxy/envoy-build-ubuntu http_proxy=http://proxy.foo.com:8080 https_proxy=http://proxy.bar.com:8080 ./ci/run_envoy_docker.sh <build_script_args> |
| 72 | +``` |
| 73 | + |
| 74 | +Besides `http_proxy` and `https_proxy`, maybe you need to set `go_proxy` to replace the default GOPROXY in China. |
| 75 | + |
| 76 | +```bash |
| 77 | +IMAGE_NAME=envoyproxy/envoy-build-ubuntu go_proxy=https://goproxy.cn,direct http_proxy=http://proxy.foo.com:8080 https_proxy=http://proxy.bar.com:8080 ./ci/run_envoy_docker.sh <build_script_args> |
| 78 | +``` |
| 79 | + |
| 80 | +To force the Envoy build image to be refreshed by Docker you can set `ENVOY_DOCKER_PULL=true`. |
| 81 | + |
| 82 | +```bash |
| 83 | +ENVOY_DOCKER_PULL=true ./ci/run_envoy_docker.sh <build_script_args> |
| 84 | +``` |
| 85 | + |
| 86 | +## On Linux |
| 87 | + |
| 88 | +An example basic invocation to build a developer version of the Envoy static binary (using the Bazel `fastbuild` type) is: |
| 89 | + |
| 90 | +```bash |
| 91 | +./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.dev' |
| 92 | +``` |
| 93 | + |
| 94 | +The Envoy binary can be found in `/tmp/envoy-docker-build/envoy/source/exe/envoy-fastbuild` on the Docker host. You |
| 95 | +can control this by setting `ENVOY_DOCKER_BUILD_DIR` in the environment, e.g. to |
| 96 | +generate the binary in `~/build/envoy/source/exe/envoy-fastbuild` you can run: |
| 97 | + |
| 98 | +```bash |
| 99 | +ENVOY_DOCKER_BUILD_DIR=~/build ./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.dev' |
| 100 | +``` |
| 101 | + |
| 102 | +For a release version of the Envoy binary you can run: |
| 103 | + |
| 104 | +```bash |
| 105 | +./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.release.server_only' |
| 106 | +``` |
| 107 | + |
| 108 | +The build artifact can be found in `/tmp/envoy-docker-build/envoy/source/exe/envoy` (or wherever |
| 109 | +`$ENVOY_DOCKER_BUILD_DIR` points). |
| 110 | + |
| 111 | +For a debug version of the Envoy binary you can run: |
| 112 | + |
| 113 | +```bash |
| 114 | +./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.debug.server_only' |
| 115 | +``` |
| 116 | + |
| 117 | +The build artifact can be found in `/tmp/envoy-docker-build/envoy/source/exe/envoy-debug` (or wherever |
| 118 | +`$ENVOY_DOCKER_BUILD_DIR` points). |
| 119 | + |
| 120 | +To leverage a [bazel remote cache](https://github.com/envoyproxy/envoy/tree/main/bazel#advanced-caching-setup) add the remote cache endpoint to |
| 121 | +the BAZEL_BUILD_EXTRA_OPTIONS environment variable |
| 122 | + |
| 123 | +```bash |
| 124 | +./ci/run_envoy_docker.sh "BAZEL_BUILD_EXTRA_OPTIONS='--remote_cache=http://127.0.0.1:28080' ./ci/do_ci.sh bazel.release" |
| 125 | +``` |
| 126 | + |
| 127 | +The `./ci/run_envoy_docker.sh './ci/do_ci.sh <TARGET>'` targets are: |
| 128 | + |
| 129 | +* `bazel.api` — build and run API tests under `-c fastbuild` with clang. |
| 130 | +* `bazel.asan` — build and run tests under `-c dbg --config=clang-asan` with clang. |
| 131 | +* `bazel.asan <test>` — build and run a specified test or test dir under `-c dbg --config=clang-asan` with clang. |
| 132 | +* `bazel.debug` — build Envoy static binary and run tests under `-c dbg`. |
| 133 | +* `bazel.debug <test>` — build Envoy static binary and run a specified test or test dir under `-c dbg`. |
| 134 | +* `bazel.debug.server_only` — build Envoy static binary under `-c dbg`. |
| 135 | +* `bazel.dev` — build Envoy static binary and run tests under `-c fastbuild` with clang. |
| 136 | +* `bazel.dev <test>` — build Envoy static binary and run a specified test or test dir under `-c fastbuild` with clang. |
| 137 | +* `bazel.dev.contrib` — build Envoy static binary with contrib and run tests under `-c fastbuild` with clang. |
| 138 | +* `bazel.dev.contrib <test>` — build Envoy static binary with contrib and run a specified test or test dir under `-c fastbuild` with clang. |
| 139 | +* `bazel.release` — build Envoy static binary and run tests under `-c opt` with clang. |
| 140 | +* `bazel.release <test>` — build Envoy static binary and run a specified test or test dir under `-c opt` with clang. |
| 141 | +* `bazel.release.server_only` — build Envoy static binary under `-c opt` with clang. |
| 142 | +* `bazel.sizeopt` — build Envoy static binary and run tests under `-c opt --config=sizeopt` with clang. |
| 143 | +* `bazel.sizeopt <test>` — build Envoy static binary and run a specified test or test dir under `-c opt --config=sizeopt` with clang. |
| 144 | +* `bazel.sizeopt.server_only` — build Envoy static binary under `-c opt --config=sizeopt` with clang. |
| 145 | +* `bazel.coverage` — build and run tests under `-c dbg` with gcc, generating coverage information in `$ENVOY_DOCKER_BUILD_DIR/envoy/generated/coverage/coverage.html`. |
| 146 | +* `bazel.coverage <test>` — build and run a specified test or test dir under `-c dbg` with gcc, generating coverage information in `$ENVOY_DOCKER_BUILD_DIR/envoy/generated/coverage/coverage.html`. Specify `//contrib/...` to get contrib coverage. |
| 147 | +* `bazel.msan` — build and run tests under `-c dbg --config=clang-msan` with clang. |
| 148 | +* `bazel.msan <test>` — build and run a specified test or test dir under `-c dbg --config=clang-msan` with clang. |
| 149 | +* `bazel.tsan` — build and run tests under `-c dbg --config=clang-tsan` with clang. |
| 150 | +* `bazel.tsan <test>` — build and run a specified test or test dir under `-c dbg --config=clang-tsan` with clang. |
| 151 | +* `bazel.fuzz` — build and run fuzz tests under `-c dbg --config=asan-fuzzer` with clang. |
| 152 | +* `bazel.fuzz <test>` — build and run a specified fuzz test or test dir under `-c dbg --config=asan-fuzzer` with clang. If specifying a single fuzz test, must use the full target name with "_with_libfuzzer" for `<test>`. |
| 153 | +* `bazel.compile_time_options` — build Envoy and run tests with various compile-time options toggled to their non-default state, to ensure they still build. |
| 154 | +* `bazel.compile_time_options <test>` — build Envoy and run a specified test or test dir with various compile-time options toggled to their non-default state, to ensure they still build. |
| 155 | +* `bazel.clang_tidy <files>` — build and run clang-tidy specified source files, if no files specified, runs against the diff with the last GitHub commit. |
| 156 | +* `check_proto_format`— check configuration, formatting and build issues in API proto files. |
| 157 | +* `fix_proto_format`— fix configuration, formatting and build issues in API proto files. |
| 158 | +* `format`— run validation, linting and formatting tools. |
| 159 | +* `docs`— build documentation tree in `generated/docs`. |
| 160 | + |
| 161 | +## On Windows |
| 162 | + |
| 163 | +An example basic invocation to build the Envoy static binary and run tests is: |
| 164 | + |
| 165 | +```bash |
| 166 | +./ci/run_envoy_docker.sh './ci/windows_ci_steps.sh' |
| 167 | +``` |
| 168 | + |
| 169 | +You can pass additional command line arguments to `./ci/windows_ci_steps.sh` to list specific `bazel` arguments and build/test targets. |
| 170 | +as set environment variables to adjust your container build environment as described above. |
| 171 | + |
| 172 | +The Envoy binary can be found in `${TEMP}\envoy-docker-build\envoy\source\exe` on the Docker host. You |
| 173 | +can control this by setting `ENVOY_DOCKER_BUILD_DIR` in the environment, e.g. to |
| 174 | +generate the binary in `C:\Users\foo\build\envoy\source\exe` you can run: |
| 175 | + |
| 176 | +```bash |
| 177 | +ENVOY_DOCKER_BUILD_DIR="C:\Users\foo\build" ./ci/run_envoy_docker.sh './ci/windows_ci_steps.sh' |
| 178 | +``` |
| 179 | + |
| 180 | +Note the quotations around the `ENVOY_DOCKER_BUILD_DIR` value to preserve the backslashes in the |
| 181 | +path. |
| 182 | + |
| 183 | +If you would like to run an interactive session to keep the build container running (to persist your local build environment), run: |
| 184 | + |
| 185 | +```bash |
| 186 | +./ci/run_envoy_docker.sh 'bash' |
| 187 | +``` |
| 188 | + |
| 189 | +From an interactive session, you can invoke `bazel` directly, or use the `./ci/windows_ci_steps.sh` script to build and run tests. |
| 190 | +Bazel will look for .bazelrc in the `${HOME}` path, which is mapped to the persistent path `${TEMP}\envoy-docker-build\` on the |
| 191 | +Docker host. |
| 192 | + |
| 193 | +# Testing changes to the build image as a developer |
| 194 | + |
| 195 | +The base build image used in the CI flows here lives in the [envoy-build-tools](https://github.com/envoyproxy/envoy-build-tools) |
| 196 | +repository. If you need to make and/or test changes to the build image, instructions to do so can be found in |
| 197 | +the [build_container](https://github.com/envoyproxy/envoy-build-tools/blob/main/build_container/README.md) folder. |
| 198 | +See the Dockerfiles and build scripts there for building a new image. |
| 199 | + |
| 200 | +# macOS Build Flow |
| 201 | + |
| 202 | +The macOS CI build is part of the [Azure Pipelines](https://dev.azure.com/cncf/envoy/_build) workflow. |
| 203 | +Dependencies are installed by the `ci/mac_ci_setup.sh` script, via [Homebrew](https://brew.sh), |
| 204 | +which is pre-installed on the [Azure Pipelines macOS image](https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md). |
| 205 | +The dependencies are cached and re-installed on every build. The `ci/mac_ci_steps.sh` script executes the specific commands that |
| 206 | +build and test Envoy. Note that the full version of Xcode (not just Command Line Tools) is required. |
0 commit comments