Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add retry loop for http requests to github #76

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sheridancbio
Copy link
Contributor

  • detect when ratelimit has been triggered

This script is no longer intended for use in automated pipeline runs, but may still be useful for building the docker image for the database transfer tools. That build will (in the future) likely contain a number of baked in versions of the derivation scripts for use with various schema versions of cBioPortal databases (so that the latest image will work with previous versions of the database schema as well as the latest database schema).

This script can be used to assemble the various known versions of the clickhouse.sql file and associated files.

The change in this PR introduces 2 things:

  1. up to 5 tries will be made for fetches from github through the github content download API
  2. the "ratelimit" condition is now detected and reported to the user with intelligible warnings/errors. If the ratelimit reset is approaching (within 5 minutes) the program will wait for the reset to happen before reattempting download.

- detect when ratelimit has been triggered
@sheridancbio sheridancbio added the enhancement New feature or request label Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant