Is it possible for extras to not contain the install_requires ?

[tchaton]

Dear @jaraco,

I have been trying to find a solution to the following problem.

Here is what I would like to be able to do:

install_requires = ["A", "B", "C"]
extras = {
    "my_extra_a": ["D", "E", "G"] + (["A", "B", "C"] coming from install_requires),
    "my_base_extra": ["A"] + (prevent install_requires to be added.)
}

For production reason, we would like to be able to release the minimal version possible of our library, which is more restrictive from the current dev env. However, changing the base would be a big breaking change for the users and we don't want to take this route.

Ideally, it would be great for setuptools to be more flexible and enable fully independent installs which is more appropriate for production vs dev requirements.

install_requires = {
    "": default ones,
    "a": Whatever I want
    "b": Whatever else I want
}

Right now, I am investigating if it is possible to implement my own Distribution / Command to prevent the install_requires to be added to one of the extras. But I haven't found yet where the addition is being done.

In case it is not possible, the only other solution would be to release a second version of our library with trimmed dep, but I would prefer not to do so.

Best,
T.C

[tchaton]

I also tried the following, but I am not sure to fully understand the behaviour from setup with extras.

extras = {
    # 'docs': load_requirements(file_name='docs.txt'),
    "base": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="base.txt"),
    "examples": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="examples.txt"),
    "loggers": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="loggers.txt"),
    "extra": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="extra.txt"),
    "test": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="test.txt"),
    "": setup_tools._load_requirements(path_dir=_PATH_REQUIRE, file_name="requirements.txt"),
}
extras["dev"] = extras["extra"] + extras["loggers"] + extras["test"] + extras[""]
extras["all"] = extras["dev"] + extras["examples"]  # + extras['docs']

setup(
     ...
    setup_requires=[],
    install_requires=[]
    extras_require=extras,

Somehow, pip install -e . install extras[""] which is the expected behaviour. But pip install -e '.[base]' also adds extras[""] which is what I want to avoid.

Best,
T.C

[webknjaz]

Setuptools doesn't install stuff. pip does. This is a question for that project. But to answer your question, pip doesn't currently have a way to skip installing the unconditional dependencies.

P.S. You really should be using something like pip-tools to manage your env pins. There's a difference between the library deps and the reproducible environment setup: https://packaging.python.org/discussions/install-requires-vs-requirements/#install-requires-vs-requirements-files.

[tchaton]

Dear @webknjaz

Yes, pip install the req, but Setuptools provide the interface with install_requires, extra_requires. I wondered if I could overrides this to prevent install_requires to be added to all extra_requires.

Best,
T.C

[Borda]

Thank you, just to clarify if we use the same package later in the Conda registry would the pip-tools enhancement be sufficient or do we need to make another layer for Conda, eventually any other registry?

[webknjaz]

@tchaton setuptools merely creates artifacts per the spec. Direct dependencies are separate from the extras: https://packaging.python.org/specifications/core-metadata/#requires-dist-multiple-use. Maybe pip does not process those correctly but FWIW an empty string is a corner case. Besides, [""] probably gets interpolated by bash into [] before it even reaches pip, I don't think that putting the quotes there is supported syntax anyway. You've probably found a bug that needs to error out explicitly.

@Borda I know next to nothing about the conda ecosystem. But if it uses pip, why not?

[jaraco]

An extra named "" gets treated as an "install_requirement" (installed in all cases). That's mainly because of the syntax of requirements and the fact that Setuptools historically allowed for empty extras to be used for environment-markers. e.g.:

extras_require = {
    '; python_version<"2.7"': ['py26req'],
}

There are open feature requests to provide a "default extras" (#1139), which I believe is what you want, but that feature doesn't exist yet.