builder 3.0.0 ansible-core gets updated in user-pip requirements are installed - specifically molecule

[rfrancis2823]

ansible-builder will seemingly inadvertently update ansible-core when molecule gets installed in the user-pip requirements file:

Dependencies in v3 builder yml config:

  ansible_core:
    package_pip: ansible-core==2.11.11
  ansible_runner:
    package_pip: ansible-runner==2.1.4
  python_interpreter:
    package_system: "python38"
    python_path: "/usr/bin/python3.8"
  galaxy: aee-galaxy-requirements.yml
  python: aee-pip-requirements.txt
  system: aee-system-pkgs.txt

ansible-core version after build completes:

[root@dfe4c34434ba /]# ansible --version
ansible [core 2.11.11]

Step is in builder build stage:

RUN $PYCMD /output/scripts/introspect.py introspect --sanitize --user-pip=requirements.txt --user-bindep=bindep.txt --write-bindep=/tmp/src/bindep.txt --write-pip=/tmp/src/requirements.txt

Introspect has ansible-core in EXCLUDE_REQUIREMENTS, but this leaves open the possibility that ansible-could get updated by getting caught in dependency resolution. Molecule's ansible-compat dependency being an example:

├── molecule [required: >=3.4.0, installed: 3.5.2]
│   ├── ansible-compat [required: >=0.5.0, installed: 3.0.2]
│   │   ├── ansible-core [required: >=2.12, installed: 2.13.11]
│   │   │   ├── cryptography [required: Any, installed: 36.0.1]
│   │   │   │   └── cffi [required: >=1.12, installed: 1.15.0]
│   │   │   │       └── pycparser [required: Any, installed: 2.21]
│   │   │   ├── Jinja2 [required: >=3.0.0, installed: 3.0.3]
│   │   │   │   └── MarkupSafe [required: >=2.0, installed: 2.0.1]
│   │   │   ├── packaging [required: Any, installed: 21.3]
│   │   │   │   └── pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.6]
│   │   │   ├── PyYAML [required: >=5.1, installed: 5.4.1]
│   │   │   └── resolvelib [required: >=0.5.3,<0.9.0, installed: 0.5.4]

├── ansible-compat [required: >=0.5.0, installed: 3.0.2]
│   ├── ansible-core [required: >=2.12, installed: 2.13.11]
│   │   ├── cryptography [required: Any, installed: 36.0.1]
│   │   │   └── cffi [required: >=1.12, installed: 1.15.0]
│   │   │       └── pycparser [required: Any, installed: 2.21]
│   │   ├── Jinja2 [required: >=3.0.0, installed: 3.0.3]
│   │   │   └── MarkupSafe [required: >=2.0, installed: 2.0.1]
│   │   ├── packaging [required: Any, installed: 21.3]
│   │   │   └── pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.6]
│   │   ├── PyYAML [required: >=5.1, installed: 5.4.1]
│   │   └── resolvelib [required: >=0.5.3,<0.9.0, installed: 0.5.4]

EXCLUDE_REQUIREMENTS can be overridden by also putting the ansible-core version you want in your user requirements file, so there is a workaround. However, since there is a seprate key in the dependencies structure for ansible-core and ansible-runner, I don't think introspect should exclude them if they are defined.

Proposing that builder should append the requirements if ansible core and runner package_pip versions are specified in dependencies. Its not intuitive that it would be allowed to change since it was installed at an earlier stage.

[sivel]

We've recently talked about this scenario while specing out #664

We've decided this is not something we plan to address. Exclusions will continue to only apply to top-level deps. If the EE definition or a collection requirements.txt has indirect dependencies, they will need to be excluded, and explicit constraints applied in the EE definition.

We did talk about the potential of using piptools, and building a full dependency tree, but are not prepared to support such a scenario or the complexities of doing so.

[rfrancis2823]

Appreciate the update. That’s understandable. After spending more time maintaining our organization’s image, I completely agree with that decision. We have a pretty complex set of dependencies and trying to manage that on the builder side I think would be an unwinnable battle. Cheers!

…

________________________________ From: Matt Martz ***@***.***> Sent: Tuesday, March 26, 2024 3:46:19 PM To: ansible/ansible-builder ***@***.***> Cc: rfrancis2823 ***@***.***>; Author ***@***.***> Subject: Re: [ansible/ansible-builder] builder 3.0.0 ansible-core gets updated in user-pip requirements are installed - specifically molecule (Issue #594) We've recently talked about this scenario while specing out #664<#664> We've decided this is not something we plan to address. Exclusions will continue to only apply to top-level deps. If the EE definition or a collection requirements.txt has indirect dependencies, they will need to be excluded, and explicit constraints applied in the EE definition. We did talk about the potential of using piptools, and building a full dependency tree, but are not prepared to support such a scenario or the complexities of doing so. — Reply to this email directly, view it on GitHub<#594 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC5XZULE7NFBCC5CTPO7CNTY2HNBXAVCNFSM6AAAAAA3QPCN7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRRGQZTOOBUGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>