Description
Hello,
Ive used this image in the past, and decided to re-visit it for a new task.
I am attempting to create an interface where images are uploaded via a POST request, perform inference with a yolov3 model, and return some JSON at the end.
I have tested the implementation with a local flask server via flask run, and tested with the base python:3.8 image, both of which run successfully, returning results at an acceptable rate.
As soon as I try to use this image, or the tiangolo/meinheld-gunicorn-flask images, requests to the flask server are incredibly slow. Increasing the timeout enough, I find that eventually it works, returning a response almost 5 minutes later, which is unfortunate.
I believe the issue stems from buffering file contents somewhere between nginx and uwsgi, as I see a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000001.
I have not attempted to test concurrent requests yet, as it is struggling to handle a single one already.
I have tried various custom configurations for various parameters from http://nginx.org/en/docs/http/ngx_http_uwsgi_module.html, adding with COPY buffer.conf /etc/nginx/conf.d/ in the Dockerfile, without much success, and was hoping to get some guidance about how to configure this.
The flask endpoint is designed as follows:
@app.route('/detect', methods=['POST'])
def detection():
try:
print('Reading image...')
image = request.files['image'].read()
nparr = np.frombuffer(image, np.uint8)
cwt_image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
boxes = detection.getBoundingBoxes(cwt_image)
detections = []
for box in boxes:
detections.append(box)
response = {
'detections' : detections
}
except Exception as e:
error = 'error: {}'.format(repr(e))
print(error)
response = (error, 500, [])
return response
And I am testing by sending a request to the endpoint as follows:
file = {'image': open('image', 'rb')}
response = requests.post(url=addr, files=file)
My Dockerfile is as follows:
FROM tiangolo/uwsgi-nginx-flask:python3.8
RUN apt-get update -y
RUN apt-get install ffmpeg libsm6 libxext6 -y
RUN pip install opencv-python
COPY buffer.conf /etc/nginx/conf.d/
ADD app /app
WORKDIR /app
I wonder if it is related to how I am handling the image data? The images are typically around 100-150KB in size.
Any guidance to configuring this (or any other) image properly to be able to handle concurrent requests of this nature would be great. Thanks in advance.