Markers in lock file are not respected

[johnthagen]

Follow on of #45

When a lock file contains:

[package.dependencies]
pywin32 = {version = ">=1.0", markers = "sys_platform == \"win32\""}
traitlets = "*"

When installed via tox-poetry-dev-dependencies on macOS, it still tries to feed that to tox and the install fails.

ERROR: Could not find a version that satisfies the requirement pywin32==228 (from versions: none)
ERROR: No matching distribution found for pywin32==228

[johnthagen]

The same example lock file found here triggers this: #45 (comment)

[sinoroc]

Thanks for the report.

Yes, this was bound to happen. I want to point to this comment again:

• Experimental lock file integration fails with certain dependencies #45 (comment)

I will look into it anyway and if the amount of new code required is reasonable I will fix it. Otherwise I would be inclined to wait and see if and when the lockfile handling logic is added to poetry-core, so that that we can reuse it.

[johnthagen]

@sinoroc Totally understand waiting for upstream support. I am able to use poetry_add_dev_dependencies = True, which provides a lot of what we need for now.

[sinoroc]

@johnthagen I guess it's also a good occasion to mention the other 2 plugin alternatives (I found so far) that I listed at the end of the readme. Maybe they are more mature, and suit your needs better.

[sinoroc]

It's a bit less straightforward than I initially expected... As far as I can see the markers info is not in the lockfile. Looks like we might need to combine info from both pyproject.toml and poetry.lock. Or am I missing something?

[sinoroc]

With poetry<1.1 the markers seem to be in the lockfile.
(I guess the lockfile format changed in version 1.1).

[johnthagen]

Yeah, in my example I saw the markers in the lockfile directly. I was using poetry 1.1.3.

Did you meant poetry>=1.1?

Or maybe my lock file was built using an older poetry and still has some old information in it.

[johnthagen]

When I lock using poetry 1.1.3. it seems to add the markers to the lock file.

[sinoroc]

With the same pyproject.toml:

• Poetry version 1.0.10:
  • I do see at least one marker in poetry.lock.
• Poetry version 1.1.3:
  • I do not see any marker in poetry.lock.

Maybe I am doing something wrong.

[johnthagen]

Maybe try ipython as a dependency, as this has several markers:

[[package]]
name = "ipython"
version = "7.18.1"
description = "IPython: Productive Interactive Computing"
category = "main"
optional = false
python-versions = ">=3.7"

[package.dependencies]
appnope = {version = "*", markers = "sys_platform == \"darwin\""}
backcall = "*"
colorama = {version = "*", markers = "sys_platform == \"win32\""}
decorator = "*"
jedi = ">=0.10"
pexpect = {version = ">4.3", markers = "sys_platform != \"win32\""}
pickleshare = "*"
prompt-toolkit = ">=2.0.0,<3.0.0 || >3.0.0,<3.0.1 || >3.0.1,<3.1.0"
pygments = "*"
traitlets = ">=4.2"

[package.extras]
all = ["Sphinx (>=1.3)", "ipykernel", "ipyparallel", "ipywidgets", "nbconvert", "nbformat", "nose (>=0.10.1)", "notebook", "numpy (>=1.14)", "pygments", "qtconsole", "requests", "testpath"]
doc = ["Sphinx (>=1.3)"]
kernel = ["ipykernel"]
nbconvert = ["nbconvert"]
nbformat = ["nbformat"]
notebook = ["notebook", "ipywidgets"]
parallel = ["ipyparallel"]
qtconsole = ["qtconsole"]
test = ["nose (>=0.10.1)", "requests", "testpath", "pygments", "nbformat", "ipykernel", "numpy (>=1.14)"]

These were added for me using 1.1.3, but I agree it is pretty mysterious why we are having opposite results.

[johnthagen]

Maybe try poetry update vs poetry lock and see if that changes anything?

[sinoroc]

@johnthagen This is an excerpt of poetry.lock generated by Poetry version 1.0.10:

[[package]]
category = "main"
description = "Python for Window Extensions"
marker = "sys_platform == \"win32\""
name = "pywin32"
optional = false
python-versions = "*"
version = "228"

Notice the marker line. This is the kind of lines that would be helpful to us for our current issue.

But for Poetry version 1.1.3, I do not see that kind of marker information:

[[package]]
name = "pywin32"
version = "228"
description = "Python for Window Extensions"
category = "main"
optional = false
python-versions = "*"

[johnthagen]

It looks like the information I see is in [[package.dependencies]], rather than [[package]], so it looks like it's specified for transitive dependencies.

In my case, it was juypter-core:

# poetry.lock
[[package]]
name = "jupyter-core"
version = "4.6.3"
description = "Jupyter core package. A base package on which Jupyter projects rely."
category = "main"
optional = false
python-versions = "!=3.0,!=3.1,!=3.2,!=3.3,!=3.4,>=2.7"

[package.dependencies]
pywin32 = {version = ">=1.0", markers = "sys_platform == \"win32\""}
traitlets = "*"

[sinoroc]

@johnthagen So you can confirm my observation, right? Knowing the markers on the dependencies of the dependencies is not really useful, in our case. I think we really do need the marker line for a [[package]] section directly.

[johnthagen]

@sinoroc Ah, I see. Yes I could see this being very challenging for a third party plugin to handle.

[sinoroc]

Current status

Blocked until:

• either the lockfile logic is moved to poetry-core
• or the markers are added back to the lockfile

Although it would be possible, I do not think it is sensible to duplicate the lockfile logic in this plugin.

[johnthagen]

Edit: This turned out to be strictly a PyCharm bug handling the requires tox field.

This is not an issue with this library, per se, but incase someone else stumbles across this, PyCharm's tox runner currently is not compatible with tox-poetry-dev-dependencies: https://youtrack.jetbrains.com/issue/PY-45094