Screen Output Stopping Issue Due to Lack of Load-Balancing in socket.io Sessions

[Haruki112]

Describe the bug：
The implementation of stream output has been completed. However, the socket.io session lacks a load-balancing mechanism, causing significant processing speed degradation when sending a large volume of events. Additionally, screen output frequently stops.

To Reproduce：
Steps to reproduce the behavior:

1. Implement stream output.
2. Send a high volume of events continuously.
3. Monitor the screen output.

Expected behavior：
Expected continuous stream output without significant degradation in processing speed, even with high event volume.

Screenshots：

Desktop：
OS: Mac, Windows
Browser: Chrome, Edge, Safari
Version: Chainlit 1.1.404, 1.1.304

Additional context：
After reviewing the code, it appears that the socket.io session lacks a load-balancing mechanism.
Could you please provide guidance on how to address this?

Additionally, I would like assistance with adding a load-balancing mechanism to the socket.io session.

[dokterbob]

Thanks for the feedback.

Could you please share a minimal example and/or whatever tooling you used for your stress testing?

Also, please share any measurements of what was going on your server (particularly CPU load, RAM usage, disk I/O). It might be that you just need to scale-up your deployment (e.g. have more instances running or beefier metal etc.).

[Haruki112]

@dokterbob
I ran Chainlit usingchainlit run app.pyand sent multiple text-based questions.

[dokterbob]

Thanks @Haruki112, please share a main.py and any specific steps etc. required to replicate.