LaraLlama
Handling rate limits for API requests #14
Unanswered
ScottBlinman
asked this question in
Q&A
Replies: 2 comments
-
|
wow just noticed this! I am using Horizon to mange this. There is a new Horizon queue for OpenAi that I only allow 3 jobs at a time. Same with Ollama. Hope it is working well! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I have managed to get this system working on a Ubuntu server. Any ideas on how to handle the rate limits for OpenAI?
Is there a way we can use batching for the API requests that are sent?
Keen to discuss further
Beta Was this translation helpful? Give feedback.
All reactions