context_length_exceeded error on gtp-4-32k at 8193 tokens

[brandco]

We are trying to use gpt-4-32k with chatbot-ui but we are getting errors when over 8193 tokens.
We are using Azure OpenAI API and running it in a Chatbot-ui in a Docker container. The model gpt-4-32k is selectable in the dropdown when starting a new chat.

Looking at the metrics in Azure, it doesn't look like Chatbot-ui is using gpt-4-32k, but rather gpt-4, which would explain the error.

I have specified the environment variable AZURE_DEPLOYMENT_ID=gpt-4 in docker run...
It looks like this is hardcoding the API url to only ever use that deployment. So why does the drop down show gpt-4-32k as an option and having no effect?

Could we disable the dropdown option if it's not possible to change the deployment in Azure, or better yet, allow more than one deployment to be specified via environment variables and use the deployments supplied to switch the populate the model selector drop down. (I'd rather not set it up to run only gpt-4-32k because of the costs.)

[itbm]

PRs #508 and #509 fix this. PR #507 is also useful for GPT-4-32K otherwise responses are limited to 1000 tokens.