Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update AWS CodeArtifact documentation to cover [[tool.uv.index]] API #9867

Open
AdrianB-sovo opened this issue Dec 13, 2024 · 10 comments · May be fixed by #10826
Open

Update AWS CodeArtifact documentation to cover [[tool.uv.index]] API #9867

AdrianB-sovo opened this issue Dec 13, 2024 · 10 comments · May be fixed by #10826
Labels
documentation Improvements or additions to documentation help wanted Contribution especially encouraged

Comments

@AdrianB-sovo
Copy link

Currently, when using UV_INDEX and AWS CodeArtifact, a [[tool.uv.index]] section is automatically added in pyproject.toml (e.g. when calling uv add pandas) with the URL that includes the token, which mean I manually have to remove it before doing a git commit.

Here an example of lines generated in pyproject.toml when using AWS CodeArtifact:

[[tool.uv.index]]
url = "https://aws:<token>@somecompany-123456789012.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/"

Could there be a way to avoid that?

BTW: the doc https://docs.astral.sh/uv/guides/integration/alternative-indexes/#aws-codeartifact still uses the deprecated UV_EXTRA_INDEX_URL instead of UV_INDEX.

@FishAlchemist
Copy link
Contributor

UV can be configured to set secrets in environment variables, but I'm unsure if this approach is suitable for AWS.
https://docs.astral.sh/uv/configuration/indexes/#providing-credentials

e.g.

[[tool.uv.index]]
name = "aws-code"
url = "https://somecompany-123456789012.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/"
export UV_INDEX_AWS_CODE_USERNAME=aws
export UV_INDEX_AWS_CODE_PASSWORD=<token>

I see that I've modified the twine section in this way. Perhaps I haven't made a mistake?
https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-twine.html#python-configure-twine-without-login

[distutils]
index-servers =
 codeartifact
[codeartifact]
repository = https://my_domain-111122223333.d.codeartifact.us-west-2.amazonaws.com/pypi/my_repo/
password = auth-token
username = aws

@AdrianB-sovo
Copy link
Author

Thanks, it does work! 😃
Maybe it should be the recommended method described in the documentation?

@zanieb
Copy link
Member

zanieb commented Dec 13, 2024

Yeah the index integration docs were community contributed before we supported UV_INDEX — we should update those to cover both modes.

@zanieb zanieb added the documentation Improvements or additions to documentation label Dec 13, 2024
@zanieb zanieb changed the title Section [[tool.uv.index]] added when using UV_INDEX and AWS CodeArtifact Update AWS CodeArtifact documentation to cover [[tool.uv.index]] API Dec 13, 2024
@FishAlchemist
Copy link
Contributor

@zanieb Should we update the password authentication part of Google Artifact Registry at the same time?

@zanieb
Copy link
Member

zanieb commented Dec 13, 2024

Yeah it makes sense to update it all — I don't feel strongly about separate pull requests vs one.

@lmanc
Copy link

lmanc commented Dec 23, 2024

I was about to open an identical issue, but it seems that writing the index in [[tool.uv.index]] is intentional.

The workaround I'm currently using is to set it to UV_EXTRA_INDEX_URL:

export UV_EXTRA_INDEX_URL="https://aws:${AWS_CODEARTIFACT_TOKEN}@${AWS_DOMAIN}-${AWS_ACCOUNT_ID}.d.codeartifact.${AWS_REGION}.amazonaws.com/pypi/${AWS_CODEARTIFACT_REPOSITORY}/simple/"

When I run uv add <package>, it says:

$ uv add --dev "pytest>=8.3.4"
warning: Indexes specified via `--extra-index-url` will not be persisted to the `pyproject.toml` file; use `--index` instead.

which is exactly what I want, as UV_EXTRA_INDEX_URL contains a secret.

@FishAlchemist suggested a way to keep the index URL secret while using the new UV_INDEX configuration: https://docs.astral.sh/uv/configuration/indexes/#providing-credentials.

However, after reading the page, I'm unsure how to set it up with this method.

According to https://docs.astral.sh/uv/guides/integration/alternative-indexes/#aws-codeartifact I have:

  • AWS_DOMAIN
  • AWS_ACCOUNT_ID
  • AWS_ACCOUNT_ID
  • AWS_CODEARTIFACT_REPOSITORY
  • AWS_CODEARTIFACT_TOKEN

Where should I put each one of these, using the other method? It feels like I'm missing something trivial.

@lmanc
Copy link

lmanc commented Dec 23, 2024

I think I figured it out, maybe...?

[[tool.uv.index]]
name = "custom-name"
url = "https://<AWS_DOMAIN>-<AWS_ACCOUNT_ID>.d.codeartifact.<AWS_REGION>.amazonaws.com/pypi/<AWS_CODEARTIFACT_REPOSITORY>/simple/"

Where <> are placeholders, not environment variables.

Then, in .bashrc or wherever environment variables are defined:

export AWS_DOMAIN=
export AWS_ACCOUNT_ID=
export AWS_REGION=
export AWS_CODEARTIFACT_REPOSITORY=

export AWS_CODEARTIFACT_TOKEN="$(
    aws codeartifact get-authorization-token \
    --domain $AWS_DOMAIN \
    --domain-owner $AWS_ACCOUNT_ID \
    --region $AWS_REGION \
    --query authorizationToken \
    --output text
)"

export UV_INDEX_CUSTOM_NAME_USERNAME="aws"
export UV_INDEX_CUSTOM_NAME_PASSWORD="$AWS_CODEARTIFACT_TOKEN"

Where:

  • name = <custom-name> must match UV_INDEX_<CUSTOM_NAME>_USERNAME, written in all caps with _ replacing -
  • aws is just a placeholder for logging into CodeArtifact via a token.
  • UV_INDEX_<NAME>_PASSWORD could also be a token.

Am I right?

This is slightly counterintuitive, and I agree that it should be added to the documentation.

@Haknt
Copy link

Haknt commented Jan 6, 2025

@lmanc You seem to be on the right track, and this approach looks reasonable to me.


Here’s how I understand it could work:

Adding a Package with a Specific Index:

  1. Set Environment Variables:

    export AWS_DOMAIN=
    export AWS_ACCOUNT_ID=
    export AWS_REGION=
    export AWS_CODEARTIFACT_REPOSITORY=
    
    export AWS_CODEARTIFACT_TOKEN="$(
        aws codeartifact get-authorization-token \
        --domain $AWS_DOMAIN \
        --domain-owner $AWS_ACCOUNT_ID \
        --region $AWS_REGION \
        --query authorizationToken \
        --output text
    )"
    
    export UV_INDEX_CUSTOM_NAME_USERNAME="aws"
    export UV_INDEX_CUSTOM_NAME_PASSWORD="$AWS_CODEARTIFACT_TOKEN"
  2. Add a Package Using UV:

    uv add <PACKAGE_NAME> --index custom_name=<INDEX_URL>

Alternatively:

If you’d rather include the index URL directly, you might use a command like this:

uv add <PACKAGE_NAME> --index custom_name=https://${AWS_DOMAIN}-${AWS_ACCOUNT_ID}.d.codeartifact.${AWS_REGION}.amazonaws.com/pypi/${AWS_CODEARTIFACT_REPOSITORY}/simple/

This should add the following lines to your pyproject.toml file:

[tool.uv.sources]
<PACKAGE_NAME> = { index = "custom-name" }

[[tool.uv.index]]
name = "custom-name"
url = "<INDEX_URL>"

Using the Index Exclusively for a Single Library:

If you want the index to apply only to the specific library, you could try adding explicit = true like this:

[tool.uv.sources]
<PACKAGE_NAME> = { index = "custom-name" }

[[tool.uv.index]]
name = "custom-name"
url = "<INDEX_URL>"
explicit = true

If there’s anything I’ve misunderstood or missed, feel free to correct me. Also, I agree that adding this to the documentation would make things clearer for others encountering similar challenges. 😊

Note: <PACKAGE_NAME> and <INDEX_URL> are placeholders. Replace them with the actual package name and index URL.

@zanieb zanieb added the help wanted Contribution especially encouraged label Jan 6, 2025
@AdrianB-sovo
Copy link
Author

I still see an issue with that in my case, where we have dev and production indexes in CodeArtifact.
Currently, we have:

pyproject.toml

# ...

dependencies = [
    "my-package[aws,utils]==3.0.123",
]

[[tool.uv.index]]
name = "aws-codeartifact"
url = "https://company-123456789.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/"

When we develop, we export the two following env variables:

export UV_INDEX_AWS_CODEARTIFACT_USERNAME=aws
export UV_INDEX_AWS_CODEARTIFACT_PASSWORD="${CODEARTIFACT_TOKEN}"

This will create a uv.lock with something like:

[[package]]
name = "my-package"
version = "3.0.123"
source = { registry = "https://company-123456789.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/" }
sdist = { url = "https://company-123456789.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/my-package/3.0.123/my-package-3.0.123.tar.gz", hash = "sha256:12345..." }
wheels = [
    { url = "https://company-123456789.d.codeartifact.eu-central-1.amazonaws.com/pypi/development/simple/my-package/3.0.123/my-package-3.0.123-py3-none-any.whl", hash = "sha256:12345..." },
]

Now, I'm searching how I could instead use the production index instead when deploying into production with the CI (i.e. https://company-123456789.d.codeartifact.eu-central-1.amazonaws.com/pypi/production/simple/).

One way would be to do uv export --frozen --no-dev --no-hashes > requirements.txt, then install using pip like I would usually do before using uv, but it's not a great solution...

@mkniewallner
Copy link
Contributor

Started a draft to rework the documentation in #10826.

The main reason it's in draft is because I won't be able to test AWS CodeArtifact nor Google Artifact Registry until next week, and won't be able to test Azure Artifacts at all. But if someone is able to test the changes before that, especially for Azure Artifacts as I can't, feel free to report what works and what doesn't in the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation help wanted Contribution especially encouraged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants