pip3 pkg provider on Python 3.11.2 fails without break-system-packages

[damonbreeden]

Describe the Bug

on Python 3.11.2 (bookworm default) pip3 pkgs can't be installed without a specific flag --break-system-packages

Expected Behavior

pkgs to be installed without adding flags everywhere

Steps to Reproduce

Steps to reproduce the behavior:
install python 3.11 on bookworm
install a pkg with pip3 provider

Environment

• Version [puppet 7.29.1]
• Platform [Python 3.11.2, debian 12 bookworm]

Additional Context

Add any other context about the problem here.
i have to add this all over my codebase:

    provider        => 'pip3',
    install_options => '--break-system-packages',

[benjamir]

Convinience over safety?

1. System-wide apps on a stable Debian installation are mainly managed by apt.
2. Puppet provides the ability to manage system-wide packages via apt (why pip at all?)
3. pip is just a random app, that should not break a system installation (that's what a lot of server admins expect) -- as an admin I don't care what anybody does with "--user", go and play there
4. Sytem-wide pip on Debian is a minefield

To me the flag is anything but a bug. I consider it a careful decision that the choice made by Debian regarding pip is mirrored in puppet.
Implicit handling of the flag would assume everbody is aware about the implications for the whole OS.

IMO should be closed as "invalid".

[damonbreeden]

When we use puppet to configure a server the assumption is that I, as the one configuring puppet, know the ramifications of installing packages, from whatever source. The other assumption is that I’m competent enough to recover when things go sideways. Maybe just a flag we can set in hiera would solve for both of us: I don’t have to whitelist every app individually, and you can have plausible deniability when somebody’s systems blow up? It occurs to me that maybe this isn’t even the right forum for this discussion- is the pip package provider from puppet or is it part of the python module?

…

On Jul 4, 2024 at 4:48 PM -0500, Benjamin ***@***.***>, wrote: Convinience over safety? 1. System-wide apps on a stable Debian installation are mainly managed by apt. 2. Puppet provides the ability to manage system-wide packages via apt (why pip at all?) 3. pip is just a random app, that should not break a system installation (that's what a lot of server admins expect) -- as an admin I don't care what anybody does with "--user", go and play there 4. Sytem-wide pip on Debian is a minefield To me the flag is anything but a bug. I consider it a careful decision that the choice made by Debian regarding pip is mirrored in puppet. Implicit handling of the flag would assume everbody is aware about the implications for the whole OS. IMO should be closed as "invalid". — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[joshcooper]

It occurs to me that maybe this isn’t even the right forum for this discussion- is the pip package provider from puppet or is it part of the python module?

The pip3 (and related) providers are in puppet https://github.com/puppetlabs/puppet/blob/main/lib/puppet/provider/package/pip3.rb There are separate modules for managing pip and virtual environments like https://github.com/voxpupuli/puppet-python/blob/master/examples/pip.pp

expected behavior: pkgs to be installed without adding flags everywhere

Are you suggesting we add --break-system-packages by default? That seems like not a good idea in general. Can you provide more details about what package you're installing and the error you get?

[joshcooper]

@damonbreeden it's possible you were running into new behavior in pip3 that broke how puppet detects installed packages. We updated puppet's pip3 provider in #9481 Can you test a puppet7 nightly build from https://nightlies.puppet.com/apt to verify --break-system-packages is no longer needed.

[damonbreeden]

hi, i'm sorry, maybe i didn't explain the behavior correctly
this is new behavior in pip3 (externally-managed-environment):

$ pip3 install nano
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

i'm suggesting that puppet adds a flag that i can use to always add --break-system-packages to our pip3 pkgs

[AriaXLi]

Hi @damonbreeden, thank you for reporting this issue. We do not anticipate addressing this as it represents a technical direction that we have decided not to follow at Puppet. As such, this issue will be closed as “Won’t Do”. If any watcher believes this is an error, please add a comment explaining.