See #3499 for the config. Note: the screenshot incorrectly shows me entering the accounts.google.com:oaud as accounts.google.com:aud in the web UI. If you use the AWS IAM litellm-google-demo role trust policy I gave as an example, you'll overwrite what the web UI made anyways. :)

@ishaan-jaff re: d77aea7

You need to add this as an OIDC provider on the AWS IAM side first:

https://oidc.circleci.com/org/c5a99188-154f-4f69-8da2-b442b1bf78dd

After that, we can create a role that matches on both the v1 and v2 token. What branches do you want to allow accessing your Bedrock account?

The AWS IAM trust policy should be like for CircleCI V2:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::123456789012_REPLACE_ME:oidc-provider/oidc.circleci.com/org/c5a99188-154f-4f69-8da2-b442b1bf78dd"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "oidc.circleci.com/org/c5a99188-154f-4f69-8da2-b442b1bf78dd:sub": "org/c5a99188-154f-4f69-8da2-b442b1bf78dd/project/*/user/*/vcs-origin/github.com/BerriAI/litellm/vcs-ref/refs/heads/*"
                }
            }
        }
    ]
}

BerriAI might need to be lowercase, not 100% sure. You can replace the last wildcard with like "main" if you only want to allow the policy to allow running on the main branch instead of all.

And that same AWS IAM role can have these inline permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Resource": "*"
        }
    ]
}

Side note: You could restrict/deny access to specific model resources if needed, though I'm not sure that's what you'd want to do for test infrastructure like this. 🙂 https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference