[Bug]: Prompt token count of 5677 exceeds max token count of 4095.

[SamWylde]

What happened?

Using any of the gpt-4 models, if the tokens exceed 4095, we get an error. Gpt-4 models should support 128,000 tokens.

"Something went wrong. Here's the specific error message we encountered: Prompt token count of 5677 exceeds max token count of 4095."

Steps to Reproduce

1. Start a new conversation (gpt-4-turbo-preview is used for this example).
2. Use this prompt: https://docs.google.com/document/d/1tyRjbxhual4JIYUgL6kERDLwU4lfuF2BG56kIZ35Trc

What browsers are you seeing the problem on?

Firefox

Relevant log output

No response

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Try switching the “agent” model

[SamWylde]

Thank you for the help Danny!

Unfortunately I still get the same error.

I also tried: gpt-4, gpt-4-0125-preview, and gpt-4-1106-preview.

[danny-avila]

I’m not sure why the “Plugins” endpoint is giving you this issue but the “OpenAI” endpoint is working fine.

I will look into it, but switching the endpoint is a “quick fix”

[abhranil26]

i am using groq api and using llama 3 70b which has max tokens limit of 9000 but librechat gives me this error
2024-05-02 05:39:12 error: [handleAbortError] AI response error; aborting request: Prompt token count of 4104 exceeds max token count of 4095.