Consider use of Pipenv to control Python dependencies (black, cmake-format, etc)

[JoeLoser]

In the style-check CI job, there are some checks which make sure a changeset is formatted correctly with some tools such as black and cmake-format.

In particular, I often am on the latest version of many tools, but the formatting expected from a project corresponds to an older version of the tool. For example, black frequently changes formatting even in minor upgrades.

I recommend providing a Pipfile and Pipfile.lock to improve the workflow when doing local development. With this, we can pin the exact versions used for these Python modules so I don't run into unexpected formatting differences compared to what CI is using.

[tobim]

With this, we can pin the exact versions used for these Python modules so I don't run into unexpected formatting differences compared to what CI is using.

We already have Nix based tooling to set up a full development environment including dev dependencies. I would strongly suggest we try to integrate CI formatting checks with that instead of investing effort into a problem that is essentially already solved. Having a single tool for dependency management for a polyglot codebase like we have brings a couple of advantages, especially considering python wheels often bundle precompiled C or C++ libraries which might introduce conflicts.

Only if that turns out to be unsuitable we should look at alternatives that focus on the python ecosystem specifically. The ones that come to mind are Poetry, pip-tools, and pipenv.

[dominiklohmann]

It's an issue with many facets. On the one hand, using Nix in CI would solve this perfectly because it truly locks down dependencies. On the other hand, recommending Nix for local development is just not practical, because it increases the barrier to entry by just so much.

I personally agree with the Pipfile approach. It also has Dependabot support, so that would make staying on latest a breeze.

One additional approach we could take is to add format and check-format targets In our CMake, which would then do the correct thing for the user automatically.

[dominiklohmann]

For the record: In 0d889e3 we switched to installing clang-format from Linuxbrew in CI for proper formatting of concepts.

[mavam]

We are now using Poetry for most of our Python tooling, along with Black for linting. I think that addresses the OP.