Skip to content

CI: Fix slow mamba solver issue by limiting boto3 versions #61594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

datapythonista
Copy link
Member

Closes #61531

Probably better to rerun the CI 3 or 4 times to be sure this is the problem and the solution. But based on local tests, seems like boto3 has a huge number of versions (they release almost every day), and that's the problem with the mamba solver. Limiting the number of versions provided to the solver should help. 1.27 is from 2 years ago, consistent with other packages. Why only fails for 3.13? No idea

@datapythonista datapythonista requested a review from mroeschke as a code owner June 6, 2025 21:01
@datapythonista datapythonista added CI Continuous Integration Dependencies Required and optional dependencies labels Jun 6, 2025
@datapythonista
Copy link
Member Author

This doesn't seem to work. Removing boto3 from the environment seems to fix the problem locally, I thought limiting the options would also fix it. But seems like even pinning to a specific version doesn't work. I guess the problem may be with a boto3 dependency, but not too sure.

@datapythonista
Copy link
Member Author

The dependencies for boto3 are here: https://github.com/conda-forge/boto3-feedstock/blob/main/recipe/meta.yaml#L23

Not too sure if the problem is really with boto3, maybe it's more complex but removing boto3 makes it work. But doesn't seem like mamba really checks their issues, and I don't think we want to get into the mamba solver internals ourselves, so I guess we need to just keep trying something that makes the solver for 3.13 take a reasonable time.

@datapythonista
Copy link
Member Author

Based on my tests, this is what makes the solver happy. Relaxing the conditions causes the very slow mamba install times. This breaks the check for minimum versions, so it can't be merged as is (and I'll add a comment when we decide what to do).

@mroeschke not sure if you have an opinion on what to do to avoid the mamba problems based on this. Not too sure what's going on.

@mroeschke
Copy link
Member

I'm curious if conda's libmamba solver has the same issue. Potentially based on conda/conda-libmamba-solver#668 it may. (It's on my radar to maybe switch to https://github.com/conda-incubator/setup-miniconda given that mamba activity is kinda slow as you mentioned)

It's a shame we have to pin, but I'm OK with it to speed up solve times

@datapythonista
Copy link
Member Author

I had the impression that conda, mamba, micromamba and pixi all use libmamba, so not sure if changing tool would be helpful. But I'm not so sure about the exact details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Dependencies Required and optional dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Micromamba taking too long to resolve the environments in the CI
2 participants