Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Respect visitation order for proxy packages #10833

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

charliermarsh
Copy link
Member

Summary

This PR reverts #10441 and applies a different fix for #10425.

In #10441, I changed prioritization to visit proxies eagerly. I think this is actually wrong, since it means we prioritize proxy packages above everything else. And while a proxy only depends on itself, it does mean we're selecting a version for the proxy package earlier than anything else. So, if you look at #10828, we end up choosing a version for async-timeout before we choose a version for langchain, despite the latter being a first-party dependency. (async-timeout has a marker on it, so it has a proxy package, so we solve for it first.)

To fix #10425, we instead need to make sure we visit proxies in the order we see them. I think the virtual tiebreaker for proxies is reversed? We want to visit the package we see first, first.

So, in short: this reverts #10441, then corrects the ordering for visiting proxies.

Closes #10828.

@charliermarsh charliermarsh requested a review from konstin January 21, 2025 23:15
@charliermarsh charliermarsh added bug Something isn't working resolver Related to the package resolver labels Jan 21, 2025
@konstin
Copy link
Member

konstin commented Jan 22, 2025

To fix #10425, we instead need to make sure we visit proxies in the order we see them. I think the virtual tiebreaker for proxies is reversed? We want to visit the package we see first, first.

Would it fix #10425 if we visited the virtual packages inside the package-batch first?

Like:

  • A{foo}
  • A{bar}
  • A <- This one is last
  • B <- has no virtual packages
  • C{python_version}
  • C{extra}
  • C <- this one is last

Copy link
Member

@konstin konstin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #10853 for a follow-up

Comment on lines +2304 to -2313
// Verify that the package is allowed under the hash-checking policy.
if !self
.hasher
.allows_package(candidate.name(), candidate.version())
{
return Ok(None);
}

// Emit a request to fetch the metadata for this version.
if self.index.distributions().register(candidate.version_id()) {
// Verify that the package is allowed under the hash-checking policy.
if !self
.hasher
.allows_package(candidate.name(), candidate.version())
{
return Err(ResolveError::UnhashedPackage(candidate.name().clone()));
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are the hash changes coming from?

/// Main priority and tiebreak for virtual packages
type Priority = (Option<PubGrubPriority>, u32);
/// Main priority and tiebreak for virtual packages.
type Priority = (Option<PubGrubPriority>, Option<PubGrubTiebreaker>);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #10853

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working resolver Related to the package resolver
Projects
None yet
2 participants