Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: -j4 with lin/win runners #9510

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

plowsof
Copy link
Contributor

@plowsof plowsof commented Oct 9, 2024

https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

not yet able to confirm windows builds . ( i think unrelated but im having issues with gitian for lin/win using master on my own fork)

@plowsof plowsof changed the title build.yml: -j4 with lin/win runners CI: -j4 with lin/win runners Oct 9, 2024
@jeffro256
Copy link
Contributor

What kind of issue? Adding threads probably won't make the build work better

Copy link
Collaborator

@selsta selsta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears GitHub Actions runners increased their available RAM at some point so we can build with -j4

@plowsof
Copy link
Contributor Author

plowsof commented Oct 10, 2024

What kind of issue? Adding threads probably won't make the build work better

as per selstas comment, i thought the runners where unstable (even with 16GB of ram) to explain why we use 2.. 3 threads, but they previously had only 8GB of ram so it makes sense now.

The gitian issues i see is unrelated to resources, i am trying to build master,selsta is aware that its not building as it hasn't been attempted for over a year. for reference: gitian master branch run

@iamamyth
Copy link

Out of curiosity, does anyone know the approximate RAM requirement (i.e. a fairly close upper bound)? If so, I might be willing to add a follow-on PR to auto-set the number of runners, which, aside from future auto-scaling, would serve to document the rationale for the computation.

@plowsof
Copy link
Contributor Author

plowsof commented Oct 17, 2024

@iamamyth win/linux runners have 16GB of ram with 14GB SSD of storage each. the full runner list specs can be found in the link in the OP. *sorry, the upper bound is 2GB per thread i think?

@iamamyth
Copy link

Looking through the history described in this PR, it seems 8GB didn't suffice for 4 threads, but worked for 3 threads, and 16GB works for 4 threads, so perhaps 2.75 GB/thread would be appropriate (leaves a bit of breathing room vs the 2 2/3 GB strict bound).

@iamamyth
Copy link

I got auto-configuration of the runner count working on my fork and can confirm the changes here match my above formula (at least 2.75 GiB and one core per worker).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants