-
-
Notifications
You must be signed in to change notification settings - Fork 607
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
configuring for image uploads and ML inference - issues with file buffering #282
Comments
Id just like to add that I cant seem to disable the buffers with my custom configuration settings:
I am not sure if they are being over-written somewhere? Logs from the container startup:
|
Repository owner
locked and limited conversation to collaborators
Aug 29, 2024
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Hello,
Ive used this image in the past, and decided to re-visit it for a new task.
I am attempting to create an interface where images are uploaded via a
POST
request, perform inference with a yolov3 model, and return some JSON at the end.I have tested the implementation with a local flask server via
flask run
, and tested with the basepython:3.8
image, both of which run successfully, returning results at an acceptable rate.As soon as I try to use this image, or the
tiangolo/meinheld-gunicorn-flask
images, requests to the flask server are incredibly slow. Increasing the timeout enough, I find that eventually it works, returning a response almost 5 minutes later, which is unfortunate.I believe the issue stems from buffering file contents somewhere between nginx and uwsgi, as I see
a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000001
.I have not attempted to test concurrent requests yet, as it is struggling to handle a single one already.
I have tried various custom configurations for various parameters from http://nginx.org/en/docs/http/ngx_http_uwsgi_module.html, adding with
COPY buffer.conf /etc/nginx/conf.d/
in the Dockerfile, without much success, and was hoping to get some guidance about how to configure this.The flask endpoint is designed as follows:
And I am testing by sending a request to the endpoint as follows:
My Dockerfile is as follows:
I wonder if it is related to how I am handling the image data? The images are typically around 100-150KB in size.
Any guidance to configuring this (or any other) image properly to be able to handle concurrent requests of this nature would be great. Thanks in advance.
The text was updated successfully, but these errors were encountered: