Upgrading pip as per documentation fails

[giuliop]

brew doctor output

Your system is ready to brew.

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.3.8
ORIGIN: https://github.com/Homebrew/brew
HEAD: e5f776b3e23cae8f1b6c3d1194b8329bf701965f
Last commit: 4 days ago
Core tap JSON: 05 Jul 08:13 UTC
Core cask tap JSON: 05 Jul 08:13 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 10
Homebrew Ruby: 3.3.3 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.3/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.6.0 => /usr/bin/curl
macOS: 14.5-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: N/A
Rosetta 2: false

What were you trying to do (and why)?

I was trying to update pip as per the instructions in the docs.
The instructions say to use:
python3 -m pip install --upgrade pip

What happened (include all command output)?

➜  python3 -m pip install --upgrade pip
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.

    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:

    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz

    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with

    brew install pipx

    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.

    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.

    Read more about this behavior here: <https://peps.python.org/pep-0668/>

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: python3.12 -m pip install --upgrade pip

What did you expect to happen?

I was expecting pip to update.

According to the docs, that output should be expected for non-brew-packaged Python package but not for pip.

Either there is a bug or the documentation is wrong

Step-by-step reproduction instructions (by running brew commands)

brew install python
python3 -m pip install --upgrade pip

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[woodruffw]

Hi @giuliop -- I took a look, and those docs are incorrect now that Homebrew adheres to PEP 668. Time permitting, I'll try and fix those docs in the coming days.

If you want to upgrade pip, you should do so in either a virtual environment (the venv of whatever project you're developing) or use pyenv or a similar version manager to install a non-Homebrew-managed version of Python.

[woodruffw]

I just took another look at the docs, and I think they're actually currently correct. Specifically, they say this:

Starting with [email protected], the bundled Python packages should be updated by reinstalling brewed Python. For older Python formulae, they can be updated as described below.

In other words, don't support pip install --upgrade pip starting with Python 3.12, but earlier versions should still work. However, we can probably make that notice more prominent/visible.

[woodruffw]

Homebrew/brew#17922 should further clarify the docs here.

[giuliop]

I just took another look at the docs, and I think they're actually currently correct. Specifically, they say this:

Starting with [email protected], the bundled Python packages should be updated by reinstalling brewed Python. For older Python formulae, they can be updated as described below.

In other words, don't support pip install --upgrade pip starting with Python 3.12, but earlier versions should still work. However, we can probably make that notice more prominent/visible.

Indeed, re-reading the docs now they do appear correct to me too