Skip to content

Process image lines concurrently, 10 at a time #804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

dereknola
Copy link
Member

@dereknola dereknola commented Jan 8, 2025

Pull Request Checklist

  • Change does not remove any existing Images or Tags in the images-list file
  • Change does not remove / overwrite exiting Images or Tags in Rancher DockerHub
  • If updating an existing entry, verify the SOURCE is still accurate and upstream hasn't been migrated to a new regitry or repo (if they've migrated, a new repo request to EIO is needed to comply with the SOURCE DESTINATION TAG pattern)
  • New entries are in format SOURCE DESTINATION TAG
  • New entries are added to the correct section of the list (sorted lexicographically)
  • New entries have a repo created in Rancher Dockerhub (where the image will be mirrored to)
  • New entries are licensed with Rancher favored licenses - Apache 2 and MIT - or approved licenses - as according to CNCF approved licenses.
  • New entries, when used in Rancher or Rancher's provided charts, have their corresponding origin added in Rancher's images origins file (must be added for all Rancher versions >= v2.7).
  • Changes to scripting or CI config have been tested to the best of your ability

Types of Change

Instead of sequentially processing each image line (most of which are old versions and no-op), allow 10 concurrent processes to work on a line. This should dramatically improve performance of the script times.

Linked Issues

Additional Notes

Final Checks after the PR is merged

  • Confirm that you can pull the new images and tags from DockerHub

@dereknola dereknola requested a review from a team as a code owner January 8, 2025 21:31
Copy link
Contributor

This repository uses an automated workflow to automatically label pull requests which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community pull requests better. If the pull request is still relevant, please add a comment to the pull request so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the pull request in 14 days. Thank you for your contributions.

@github-actions github-actions bot closed this Apr 6, 2025
@dereknola dereknola deleted the parallel_chunk_images branch April 30, 2025 17:44
@dereknola dereknola restored the parallel_chunk_images branch April 30, 2025 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants