-
Notifications
You must be signed in to change notification settings - Fork 19
Open
Description
rapids-s3-path is currently the source of truth for computing proper paths for CI workflow artifacts.
The problem with rapids-s3-path
is that it only returns S3 URIs (e.g. s3://rapids-downloads/ci/cudf/...
).
There are a lot of different areas where we call rapids-s3-path
and then replace s3://rapids-downloads/
with https://downloads.rapids.ai/
:
gha-tools/tools/_rapids-download-from-s3
Line 18 in 71f9e9a
s3_dl_path=$(echo "${s3_dl_path}" | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|") gha-tools/tools/rapids-upload-to-s3
Line 36 in 71f9e9a
ARTIFACTS_URL=${browsable_url/s3:\/\/${RAPIDS_DOWNLOADS_BUCKET}\//https:\/\/downloads.rapids.ai\/} ARTIFACTS_URL=$(rapids-s3-path | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|")
To prevent this manual replacement from being necessary, we should add a rapids-http-path
script that computes the https://downloads.rapids.ai
path.
The scripts should ultimately be refactored to work like this:
_rapids-path
- a new script that's called byrapids-s3-path
andrapids-http-path
to generate the URI path. This script is prefixed with an underscore (_
) to denote that it's not intended by be used publicly (only by othergha-tools
scripts).rapids-s3-path
- should be refactored to call_rapids-path
and prepend thes3://rapids-downloads/
stringrapids-http-path
- a new script that calls_rapids-path
and prepends thehttps://downloads.rapids.ai/
string
Metadata
Metadata
Assignees
Labels
No labels