error during tox's installation of tox-poetry-dev-dependencies: AttributeError: module 'poetry.core' has no attribute 'poetry'

[jclerman]

I have been using this plugin for a while (thanks!!) just for what it says on the tin: To get my pyproject.toml-defined dev-dependencies into my tox testenv.

Recently I have been encountering an error that I can't find a workaround for - and this never used to happen.

The beginning of my tox.ini looks like this:

[tox]
envlist = lint,docs,py38,py39,py310
isolated_build = True
requires = tox-poetry-dev-dependencies

[testenv]
poetry_add_dev_dependencies = True

and when I run tox I see:

.tox create: [...]
.tox installdeps: tox-poetry-dev-dependencies, tox >= 3.25.1
Traceback (most recent call last):
[...]
  File "/Users/jclerman/.pyenv/versions/3.8.7/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/jclerman/git/sirona-xyz/.tox/.tox/lib/python3.8/site-packages/tox_poetry_dev_dependencies/_hooks.py", line 245, in <module>
    def _get_poetry(project_root_path: pathlib.Path) -> poetry.core.poetry.Poetry:
AttributeError: module 'poetry.core' has no attribute 'poetry'

Looks like the plugin may no longer be compatible with recent versions of poetry, or maybe poetry-core? Is there a workaround or a recommended alternative, specifically for telling tox to include dev-dependencies?

[sinoroc]

Hi @jclerman

thanks for using my code, and thanks for this issue report.

This project has fallen quite low on my priority list, so as it is now there is very little chance I will make any new improvements (but "never say never"). This project was kind of always meant as some kind of proof of concept for this issue anyway (although it grew a bit beyond that):

• How to run tox and poetry together against multiple dependency versions python-poetry/poetry#1745
• https://stackoverflow.com/q/59377071

... and yes, it relied on some private (unstable) APIs of poetry so there was always a rather high chance of things breaking.

There are other similar projects that might be helpful, I linked them here:

• https://pypi.org/project/tox-poetry-installer/
• https://pypi.org/project/tox-poetry/

Although, as it turns out, they seem to be confronted with similar issues of incompatibility with the latest releases of poetry and/or poetry-core:

• tox-poetry-installer crashes due to import error enpaul/tox-poetry-installer#73

This other project tox-poetry-installer, seems way more active than mine, so there might be a higher chance that this gets fixed over there.

[sinoroc]

Also poetry updated their FAQ some weeks (months?) ago:

• https://python-poetry.org/docs/faq/#is-tox-supported

Maybe there is something in there for you.

[jclerman]

Thanks @sinoroc. Yes, I saw tox-poetry-installer but also noticed that it too is incompatible with the latest poetry. That plugin seems to use the dev-dependencies from the lockfile if it exists, whereas my impression is that yours uses what's specified in pyproject.toml and is therefore more in the spirit of tox, where we want to be testing against the version of dependencies that downstream users are likely to encounter - that is, latest compatible version.

In the end though, I decided that the difference between pyproject.toml and poetry.lock isn't that important when it comes to development-only dependencies, and that poetry's new dependency-groups therefore provide a way forward that doesn't require any plugins. I am now doing this (combination of isolated_build = True and commands_pre) in tox.ini, after installing poetry 1.2.0 such that it's available in the calling environment:

[tox]
envlist = lint,docs,py38,py39,py310
isolated_build = True

[testenv]
passenv = POETRY_HTTP_BASIC_xyz_PASSWORD POETRY_HTTP_BASIC_xyz_USERNAME
allowlist_externals = poetry
commands_pre =
    poetry install --only dev
commands =
    pytest {posargs}

... etc

[sinoroc]

There is this discussion happening:

• https://discuss.python.org/t/adding-a-non-metadata-installer-only-dev-dependencies-table-to-pyproject-toml/20106

If this standardization happens, then it should be safe to assume that tox would learn how to read such "dev dependencies" straight from pyproject.toml; no more need for hacks or plugin.

[sinoroc]

@jclerman (and other potential interested users), assuming that I were to find time and energy to make this plugin compatible with Poetry 1.2 and its "dependency groups"... what would you expect the UI/UX to look like (how should it be configured in tox.ini)?

Up until its version 1.2, Poetry had only the one dev-dependencies group, so we either enabled it or not with poetry_add_dev_dependencies = True in tox.ini. But now that Poetry handles many "dev" dependency groups, how should this option look like? Should it accept a list of dependency groups to install (poetry_add_dev_dependencies = test,lint)?

[jclerman]

@sinoroc: A list of dependency groups seems reasonable to me. The name of the key should probably change since it's no longer just for dev-dependencies. Maybe something like:

poetry_install_dependency_groups = 
    dev
    lint

The standardization would be great, so that tox can avoid referencing poetry directly. At the moment (lacking standardization or this plugin) after a couple false starts, I have settled on this approach in the meantime:

[tox]
isolated_build = True
...

[testenv]
allowlist_externals = poetry
commands_pre =
    poetry export --only dev --output {temp_dir}/{envname}-requirements-dev-only.txt
    pip install -r {temp_dir}/{envname}-requirements-dev-only.txt

[testenv:lint]
...

[testenv:docs]
...