[Feature]: Proxy - Allow users to set DynamoDB URL for Key Management

[ishaan-jaff]

The Feature

User wants to do key mgmt and budget tracking, but can only use DynamoDB

Motivation, pitch

user request

Twitter / LinkedIn details

No response

[ishaan-jaff]

looks like we had an issue for dynamoDB: #887

doesn't seem like dynamodb would be supported by the prisma interface - prisma/prisma#1676 (comment)

[ishaan-jaff]

An alternative here is using AWS RDS https://aws.amazon.com/rds/free/?loc=ft#Free_Tier

Which provides PostgreSQL on AWS

[krrishdholakia]

User said DynamoDB is a hard requirement.

We could just use boto3 for this - https://www.honeybadger.io/blog/using-dynamodb-with-python/

Since our db client is already abstracted away -

litellm/litellm/proxy/utils.py

Line 251 in aa757d1

class PrismaClient:

We could just do this ourselves

[krrishdholakia]

Don't think we should do this ourselves. Seems like there's different edge cases here - e.g. how you set compute cost (aws charges per read/write or by compute, etc.)

Considering using this - https://github.com/pynamodb/PynamoDB

[krrishdholakia]

I have an initial working commit - https://github.com/BerriAI/litellm/tree/litellm_dynamo_db_keys

Really concerned about the overhead of supporting 2 different db formats (nosql + sql)

[krrishdholakia]

pynamodb doesn't support async 😢 pynamodb/PynamoDB#802

exploring alternatives

[krrishdholakia]

👀 https://github.com/HENNGE/aiodynamo

[krrishdholakia]

pending work - enable spend tracking for dynamodb