Use `uv pip` or optionally support it

[crestonbunch]

pip install is used here:

buildpacks-python/src/layers/pip_dependencies.rs

Lines 81 to 89 in 0bcbf98

	Command::new("pip")
	.args([
	"install",
	"--no-input",
	"--progress-bar",
	"off",
	"--requirement",
	"requirements.txt",
	])

.

It would be great to also / alternatively use uv pip install. This is intended to be a drop-in replacement for pip, but with much faster dependency resolution and other optimizations. Ref: https://docs.astral.sh/uv/pip/

In practice, I notice a decent speed up when using uv pip install over pip install, and it would be a great improvement to have this speedup in Heroku's Python buildpack.

I think it probably makes sense as an opt-in feature with an environment variable or similar, since there's no way to distinguish when to use uv pip or pip based on the file structure alone.

[crestonbunch]

Another approach would be to support disabling requirement installation, or splitting it into a different buildpack. One advantage of the Paketo ecosystem is that it splits the CPython and pip steps into two different buildpacks, so you can bring your own requirements buildpack if you want to.

[edmorley]

Hi!

I thought about perhaps doing this when adding support for uv originally.

My reasoning for not doing so at the time, was:

1. Lockfiles are preferable to requirements files, and anyone who is actively looking to optimise their app / use best practices would be much better off switching from a requirements file to some kind of lockfile (whether that be uv's or Poetry's).
2. We of course still have to support requirements files given how widespread their usage is - and given there are several intentional behaviour differences between pip install and uv pip install, we couldn't replace pip support entirely with uv pip (as much as I would like to just drop pip). So we'd have to support both if we wanted to support uv pip.
3. Supporting both pip and uv pip would mean we'd need a way for the user to explicitly tell the buildpack to use uv pip via some kind of config option, since (as you noted) there is no official upstream way to do this (beyond say magical heuristics that could be confusing to the user, like seeing if there is a tool.uv table in pyproject.toml).
4. Therefore, since we need an explicit config option, users would have to be proactive enough to read the docs to know it exists and they might want to use it. However, I suspect many of those most proactive/aware users are going to instead switch to uv.lock instead (given (1)) - leaving very few users that might use uv pip.
5. If we support running uv in two different modes (uv sync vs uv pip), it risks users getting confused between the two, or getting confused if they run into a uv pip vs pip behaviour difference. We can try to mitigate some of this by trying to explain what's happening in the build logs, but it adds another layer of UX complexity for users. Plus user might think uv pip is "good enough", when really we want to encourage them to switch to a lockfile, per (1).
6. In terms of maintenance burden, it means another package manager permutation to have to test in CI (taking us to four permutations: pip, poetry, uv sync, uv pip), as well as needing handling for interactions between the uv modes (for example, how do we handle the case when an app has a pyproject.toml and uv.lock and is also opting into uv pip?). These aren't necessarily blockers, but do mean there need to be a fair few benefits of supporting uv pip to justify the costs.
7. In terms of advantages of uv pip over pip, whilst there are many when running uv locally, these are much reduced when running in buildpack context - since (a) much of the time is spent outside the package manager (eg uploading/downloading caches/OCI layers, or transferring bytes to/from the Docker daemon etc, (b) we'd still have to only cache the uv cache and not site-packages for the same reasons we do for pip (to prevent leftover cruft in the venv due to uv pip not syncing). So my gut feeling is that the performance benefits won't be that significant compared to the difference seen locally.

In practice, I notice a decent speed up when using uv pip install over pip install, and it would be a great improvement to have this speedup in Heroku's Python buildpack.

I presume this is locally rather than in a buildpack? Is this for legacy apps that you're not able to switch to uv.lock?

Another approach would be to support disabling requirement installation, or splitting it into a different buildpack. One advantage of the Paketo ecosystem is that it splits the CPython and pip steps into two different buildpacks, so you can bring your own requirements buildpack if you want to.

So I could write a long essay on the issues with the buildpack build plan and difficulties with buildpack composition in practice, but the short version is roughly that in many cases composing unrelated buildpacks/components doesn't work well - meaning you get the worse UX of fragmented micro-buildpacks with very few of the benefits. (The Heroku Node.js buildpacks were heavily split up until recently, and just switched to a single buildpack approach for this reason.)

[edmorley]

If you could give me an example requirements file, I could try comparing the uv pip install performance with pip install in a buildpack context?