Change server approach to handle parallel requests#1550

Open

sergey-zinchenko wants to merge 2 commits intoabetlen:mainfrom sergey-zinchenko:model_lock_per_request

+309-304

Commits on Jun 23, 2024

[model_lock_per_request] model lock rewrote to be async lock over request
Sergei Zinchenko
committed
[model_lock_per_request] added limit_concurrency for uvicorn
Sergei Zinchenko
committed