Here is a list of related issues:

Fixes:

• requirements-parser breaks on 3.12+ default venvs #644
• Dealing with broken collection deps #493
• Correctly excluding a package? #472
• Invalid requirements.txt leads to container built incorrectly #364
• Option to disable --sanitize / override collection dependencies #323
• Private python requirements impossible to install due to outdated requirements-parser #319
• Feature to ignore/sanitize some bindep requirements #201

Impacts:

• document ansible-builder introspect command #562

Hi, thanks for working on this!
Sorry for adding a comment before making enough test, but since this PR at this moment simply uses \n to separate lines for requirements.txt for collections (in pip_file_data) and process them line by line, I wonder if the collections' requirements.txt would behave unexpectedly if it contained line continuations with backslashes.

• Example collection: https://github.com/PaloAltoNetworks/pan-os-ansible/blob/develop/requirements.txt
• Docs about line continuations in requirement files: https://pip.pypa.io/en/stable/reference/requirements-file-format/#line-continuations

I wonder if the collections' requirements.txt would behave unexpectedly if it contained line continuations with backslashes

@kurokobo that is correct. We discussed this when architecting the functionality, and should a package that is excluded have a backslash with a newline, it will likely cause problems.

We don't plan to add any specific functionality to permit or deny newlines, but we do intend to add documentation that states that it is in the best interest of everyone for collection maintainers, and EE creators to use pep508 formatted requirements files. We may be going so far to add linting rules, potentially via galaxy-importer to warn when a collection defines non-pep508 formatted requirements files.

@sivel
Thanks for providing detailed background! I have no objection to the policy, but I want to know details a bit.

If we strictly follow pep508, even any comments (#) would not be allowed. But this PR does not remove an implementation that intentionally ignores comments; is it a policy that requires strictly forrowing pep508, but allows comments as an exception? Or is my understanding incorrect?

F.Y.I., to support line continuations, we need just one modification (I don't think it will be adopted, but I'll write it down as I think of it😅 ).

diff --git a/src/ansible_builder/_target_scripts/introspect.py b/src/ansible_builder/_target_scripts/introspect.py
index 553cf81..c32d121 100644
--- a/src/ansible_builder/_target_scripts/introspect.py
+++ b/src/ansible_builder/_target_scripts/introspect.py
@@ -26,7 +26,7 @@ def read_req_file(path):
 
 
 def pip_file_data(path):
-    pip_content = read_req_file(path)
+    pip_content = read_req_file(path).replace('\\\n', '')
 
     pip_lines = []
     for line in pip_content.split('\n'):

@sivel Thanks for providing detailed background! I have no objection to the policy, but I want to know details a bit.

If we strictly follow pep508, even any comments (#) would not be allowed. But this PR does not remove an implementation that intentionally ignores comments; is it a policy that requires strictly forrowing pep508, but allows comments as an exception? Or is my understanding incorrect?

Comments will be allowed. Any non-pep508 compliant entries are going to be passed through without change, unchecked and unverified. We plan to update the docs that using such non-compliant entries will be considered undefined/unsupported behavior.