Error before answer is fully generated

[aa-ko]

What happened?

Hi!

I am using LibreChat v0.7.4 to talk to a GPT-4o backend hosted at openai.azure.com.
When generating an answer, the stream stops randomly at some point with the following error message:

Something went wrong. Here's the specific error message we encountered: The response is incomplete; it's either still processing, was cancelled, or censored. Refresh or try a different prompt.

After some time, the answer is then completely replaced with the following error:

Error connecting to server, try refreshing the page.

Looking at the code in the stack trace, the error seems to be originating from the underlying OpenAI client library, but I cannot pinpoint where exactly this goes wrong.

My librechat.yml currently looks like this:

version: 1.1.5
    cache: true
    endpoints:
      azureOpenAI:
        titleModel: "gpt-4o"
        plugins: true
        groups:
        - group: "my-resource-france"
          apiKey: "${AZURE_FRANCE_API_KEY}"
          instanceName: "france"
          version: "2024-05-01-preview"
          models:
            gpt-4o:
              deploymentName: gpt4o

Any pointers on how to debug this further would be very appreciated.

Steps to Reproduce

1. Run default Docker image (ghcr.io/danny-avila/librechat-api:v0.7.4) with an Azure OpenAI endpoint configured in librechat.yml
2. Open a new chat in the web interface and ask any question

What browsers are you seeing the problem on?

Firefox

Relevant log output

[
        {
                "level": "warn",
                "message": "[OpenAIClient.chatCompletion][stream] Unhandled error type",
                "timestamp": "2024-08-19T14:02:33.443Z"
        },
        {
                "level": "warn",
                "message": "[OpenAIClient.chatCompletion][finalChatCompletion] Unhandled error type",
                "timestamp": "2024-08-19T14:02:34.177Z"
        },
        {
                "level": "error",
                "message": "[OpenAIClient.chatCompletion] Unhandled error type Error: Cannot read properties of undefined (reading 'content')",
                "stack": "Error: Error: Cannot read properties of undefined (reading 'content')\n    at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js: 1260: 15)\n    at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js: 634: 15)\n    at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js: 515: 24)\n    at async AskController (/app/api/server/controllers/AskController.js: 125: 20)\n    at async /app/api/server/routes/ask/openAI.js: 24: 5",
                "timestamp": "2024-08-19T14: 02: 34.177Z"
        },
        {
                "level": "error",
                "message": "[handleAbortError] AI response error; aborting request: Error: Cannot read properties of undefined (reading 'content')",
                "stack": "Error: Error: Cannot read properties of undefined (reading 'content')\n    at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js: 1260: 15)\n    at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js: 634: 15)\n    at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js: 515: 24)\n    at async AskController (/app/api/server/controllers/AskController.js: 125: 20)\n    at async /app/api/server/routes/ask/openAI.js: 24: 5",
                "timestamp": "2024-08-19T14: 02: 34.178Z"
        },
        {
                "code": "ERR_HTTP_HEADERS_SENT",
                "level": "error",
                "message": "[handleAbortError] error while trying to abort message Cannot set headers after they are sent to the client",
                "stack": "Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client\n    at ServerRes ponse.setHeader (node:_http_outgoing: 659: 11)\n    at ServerResponse.header (/app/api/node_modules/express/lib/response.js: 795: 10)\n    at S erverResponse.send (/app/api/node_modules/express/lib/response.js: 175: 12)\n    at ServerResponse.json (/app/api/node_modules/express/lib/re sponse.js: 279: 15)\n    at ServerResponse.send (/app/api/node_modules/express/lib/response.js: 163: 21)\n    at abortMessage (/app/api/server/ middleware/abortMiddleware.js: 30: 28)\n    at handleAbortError (/app/api/server/middleware/abortMiddleware.js: 205: 20)\n    at AskController  (/app/api/server/controllers/AskController.js: 175: 5)\n    at async /app/api/server/routes/ask/openAI.js: 24: 5",
                "timestamp": "2024-08-19T14: 02 : 34.179Z"
        }
]

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

An AI response error is usually an upstream API error. thanks for the logs, seems like azure is triggering the initial error and then it cascades into several other errors in the server. I will look into this

[aa-ko]

Thank you so much for this very quick reply! Please let me know if I can do anything to help you track this issue down.

[danny-avila]

I improved the error handling here, can you try it once it's merged?

#3710

[aa-ko]

I just manually built a Docker image from the main branch (ba9c351) and it's working perfectly fine again.
Thank you so much!

[aa-ko]

After updating to v0.7.6 I've got this exact problem again 🤔

I cannot tell if this is a regression in LibreChat, but I also cannot find anyone else having this problem, so I suspect this to be an issue in my deployment.

Since the error originated from the OpenAI client, I tried updating the dependency to 4.78.1, but the problem persists.

@danny-avila Could you provide me some hints on how to debug this and where to look? I'd be happy to contribute a fix if possible, but I am lost on how to debug this further.

[aa-ko]

After some more searching I now also found this issue in the OpenAI client repository.

This exactly fits the issue I am observing right now and is related to chat completion streaming, which leads me to believe that this is indeed the root cause of the problem in my setup.

I'll try to monitor the progress on this and try to run LibreChat with the updated dependency as soon as it has been fixed.

[danny-avila]

@aa-ko are you using docker or npm?