[Bug][Question]: does cost track for batch if I don't retrieve batch?

[yeahyung]

What happened?

I'm trying to use enterprise feature(w/ trivial key) track cost of batch.

In docs, it said LiteLLM polls the batch status until completion(https://docs.litellm.ai/docs/batches#how-cost-tracking-for-batches-api-works)

Does it mean when I(client) create batch using LiteLLM,
LiteLLM continuously poll batch status until completion and aggregates usage?

I created batch using LiteLLM, but it didn't. Instead, I retrieve batch using LiteLLM and when the batch status changed to completed, it calculate costs and store in SpendLogs Table.

Do I misunderstand the feature(docs)? Please correct me if I'm wrong. Thank you! @krrishdholakia @ishaan-jaff

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.65.4

Twitter / LinkedIn details

No response

[krrishdholakia]

Hey @yeahyung there is a background polling job but there's room for improvement there (i.e. currently it will die if proxy restarted)

Do you have redis or db setup?

[yeahyung]

Hey @yeahyung there is a background polling job but there's room for improvement there (i.e. currently it will die if proxy restarted)

Do you have redis or db setup?

@krrishdholakia I used db only.
Doesn't it support background polling job if I start LiteLLM using main method in proxy_cli.py?