Skip to content

Latest commit

 

History

History
346 lines (233 loc) · 11.7 KB

setup.md

File metadata and controls

346 lines (233 loc) · 11.7 KB
title
Setup

Setup

Run cibuildwheel locally (optional) {: #local}

Before getting to CI setup, it can be convenient to test cibuildwheel locally to quickly iterate and track down issues without even touching CI.

Install cibuildwheel and run a build like this:

# using pipx (https://github.com/pypa/pipx)
pipx run cibuildwheel

# or,
pip install cibuildwheel
cibuildwheel

You should see the builds taking place. You can experiment with options using environment variables or pyproject.toml.

!!! tab "Environment variables"

cibuildwheel will read config from the environment. Syntax varies, depending on your shell:

> POSIX shell (Linux/macOS)

```sh
# run a command to set up the build system
export CIBW_BEFORE_ALL='uname -a'

cibuildwheel
```

> CMD (Windows)

```bat
set CIBW_BEFORE_ALL='uname -a'

cibuildwheel
```

!!! tab "pyproject.toml"

If you write your options into [`pyproject.toml`](options.md#configuration-file), you can work on your options locally, and they'll be automatically picked up when running in CI.

> pyproject.toml

```
[tool.cibuildwheel]
before-all = "uname -a"
```

Then invoke cibuildwheel, like:

```console
cibuildwheel
```

Linux builds

If you've got Docker installed on your development machine, you can run a Linux build.

!!! tip You can run the Linux build on any platform. Even Windows can run Linux containers these days, but there are a few hoops to jump through. Check this document for more info.

Because the builds are happening in manylinux Docker containers, they're perfectly reproducible.

The only side effect to your system will be docker images being pulled.

macOS / Windows builds

Pre-requisite: you need to have native build tools installed.

Because the builds are happening without full isolation, there might be some differences compared to CI builds (Xcode version, Visual Studio version, OS version, local files, ...) that might prevent you from finding an issue only seen in CI.

In order to speed-up builds, cibuildwheel will cache the tools it needs to be reused for future builds. The folder used for caching is system/user dependent and is reported in the printed preamble of each run (e.g. "Cache folder: /Users/Matt/Library/Caches/cibuildwheel").

You can override the cache folder using the CIBW_CACHE_PATH environment variable.

!!! warning cibuildwheel uses official python.org macOS installers for CPython but those can only be installed globally.

In order not to mess with your system, cibuildwheel won't install those if they are
missing. Instead, it will error out with a message to let you install the missing
CPython:

```console
Error: CPython 3.9 is not installed.
cibuildwheel will not perform system-wide installs when running outside of CI.
To build locally, install CPython 3.9 on this machine, or, disable this version of Python using CIBW_SKIP=cp39-macosx_*

Download link: https://www.python.org/ftp/python/3.9.8/python-3.9.8-macosx10.9.pkg
```

Pyodide (WebAssembly) builds (experimental)

Pre-requisite: you need to have a matching host version of Python (unlike all other cibuildwheel platforms). Linux host highly recommended; macOS hosts may work (e.g. invoking pytest directly in CIBW_TEST_COMMAND is currently failing) and Windows hosts will not work.

You must target pyodide with --platform pyodide (or use --only on the identifier).

Configure a CI service

GitHub Actions [linux/mac/windows] {: #github-actions}

To build Linux, Mac, and Windows wheels using GitHub Actions, create a .github/workflows/build_wheels.yml file in your repo.

!!! tab "Action" For GitHub Actions, cibuildwheel provides an action you can use. This is concise and enables easier auto updating via GitHub's Dependabot; see Automatic updates.

> .github/workflows/build_wheels.yml

```yaml
{% include "../examples/github-minimal.yml" %}
```

Use `env:` to pass [build options](options.md) and `with:` to set
`package-dir: .`, `output-dir: wheelhouse` and `config-file: ''`
locations (those values are the defaults).

!!! tab "pipx" The GitHub Actions runners have pipx installed, so you can easily build in just one line. This is internally how the action works; the main benefit of the action form is easy updates via GitHub's Dependabot.

> .github/workflows/build_wheels.yml

```yaml
name: Build

on: [push, pull_request]

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        # macos-13 is an intel runner, macos-14 is apple silicon
        os: [ubuntu-latest, windows-latest, macos-13, macos-14]

    steps:
      - uses: actions/checkout@v4

      - name: Build wheels
        run: pipx run cibuildwheel==2.23.0

      - uses: actions/upload-artifact@v4
        with:
          name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
          path: ./wheelhouse/*.whl
```

!!! tab "Generic" This is the most generic form using setup-python and pip; it looks the most like the other CI examples. If you want to avoid having setup that takes advantage of GitHub Actions features or pipx being preinstalled, this might appeal to you.

> .github/workflows/build_wheels.yml

```yaml
name: Build

on: [push, pull_request]

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        # macos-13 is an intel runner, macos-14 is apple silicon
        os: [ubuntu-latest, windows-latest, macos-13, macos-14]

    steps:
      - uses: actions/checkout@v4

      # Used to host cibuildwheel
      - uses: actions/setup-python@v5

      - name: Install cibuildwheel
        run: python -m pip install cibuildwheel==2.23.0

      - name: Build wheels
        run: python -m cibuildwheel --output-dir wheelhouse

      - uses: actions/upload-artifact@v4
        with:
          name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
          path: ./wheelhouse/*.whl
```

Commit this file, and push to GitHub - either to your default branch, or to a PR branch. The build should start automatically.

For more info on this file, check out the docs.

examples/github-deploy.yml extends this minimal example with a demonstration of how to automatically upload the built wheels to PyPI.

Azure Pipelines [linux/mac/windows] {: #azure-pipelines}

To build Linux, Mac, and Windows wheels on Azure Pipelines, create a azure-pipelines.yml file in your repo.

azure-pipelines.yml

{% include "../examples/azure-pipelines-minimal.yml" %}

Commit this file, enable building of your repo on Azure Pipelines, and push.

Wheels will be stored for you and available through the Pipelines interface. For more info on this file, check out the docs.

Travis CI [linux/windows] {: #travis-ci}

To build Linux and Windows wheels on Travis CI, create a .travis.yml file in your repo.

.travis.yml

{% include "../examples/travis-ci-minimal.yml" %}

Commit this file, enable building of your repo on Travis CI, and push.

Then setup a deployment method by following the Travis CI deployment docs, or see Delivering to PyPI. For more info on .travis.yml, check out the docs.

examples/travis-ci-deploy.yml extends this minimal example with a demonstration of how to automatically upload the built wheels to PyPI.

AppVeyor [linux/mac/windows] {: #appveyor}

To build Linux, Mac, and Windows wheels on AppVeyor, create an appveyor.yml file in your repo.

appveyor.yml

{% include "../examples/appveyor-minimal.yml" %}

Commit this file, enable building of your repo on AppVeyor, and push.

AppVeyor will store the built wheels for you - you can access them from the project console. Alternatively, you may want to store them in the same place as the Travis CI build. See AppVeyor deployment docs for more info, or see Delivering to PyPI below.

For more info on this config file, check out the docs.

CircleCI [linux/mac] {: #circleci}

To build Linux and Mac wheels on CircleCI, create a .circleci/config.yml file in your repo,

.circleci/config.yml

{% include "../examples/circleci-minimal.yml" %}

Commit this file, enable building of your repo on CircleCI, and push.

!!! note CircleCI doesn't enable free macOS containers for open source by default, but you can ask for access. See here for more information.

CircleCI will store the built wheels for you - you can access them from the project console. Check out the CircleCI docs for more info on this config file.

Gitlab CI [linux] {: #gitlab-ci}

To build Linux wheels on Gitlab CI, create a .gitlab-ci.yml file in your repo,

.gitlab-ci.yml

{% include "../examples/gitlab-minimal.yml" %}

Commit this file, and push to Gitlab. The pipeline should start automatically.

Gitlab will store the built wheels for you - you can access them from the Pipelines view. Check out the Gitlab docs for more info on this config file.

Cirrus CI [linux/mac/windows] {: #cirrus-ci}

To build Linux, Mac, and Windows wheels on Cirrus CI, create a .cirrus.yml file in your repo,

.cirrus.yml

{% include "../examples/cirrus-ci-minimal.yml" %}

Commit this file, enable building of your repo on Cirrus CI, and push.

Cirrus CI will store the built wheels for you - you can access them from the individual task view. Check out the Cirrus CI docs for more info on this config file.

⚠️ Got an error? Check the FAQ.

Next steps

Once you've got the wheel building successfully, you might want to set up testing or automatic releases to PyPI.

<script> document.addEventListener('DOMContentLoaded', function() { $('.toctree-l3>a, .rst-content h3').each(function(i, el) { var text = $(el).text() var match = text.match(/(.*) \[([a-z/]+)\]/); if (match) { var iconHTML = $.map(match[2].split('/'), function(ident) { switch (ident) { case 'linux': return '' case 'windows': return '' case 'mac': return '' } }).join(' '); $(el).append( $('
') .append(iconHTML) .css({float: 'right'}) ) $(el).contents() .filter(function(){ return this.nodeType == 3; }).first() .replaceWith(match[1]); } }); }); </script>