In some cases, FastWSGI+Flask is faster than NGINX (latency test)

[remittor]

Client : i7-980X @ 2.8GHz, Debian 12, Python 3.10, NIC Intel x550-t2 10Gbps
Server: i7-980X @ 2.8GHz, Windows 7, Python 3.8, NIC Intel x550-t2 10Gbps

Payload for testing: https://github.com/MiloszKrajewski/SilesiaCorpus/blob/master/xml.zip (651 KiB)

Server test app: https://gist.github.com/remittor/1f2bc834852009631d437cd96822afa4

FastWSGI + Flask

python.exe server.py -h 172.16.220.205 -g fw -f xml.zip -b

$ wrk -t1 -c1 -d30 http://172.16.220.205:5000/ --latency
Running 30s test @ http://172.16.220.205:5000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.68ms  128.35us   9.26ms   97.39%
    Req/Sec   597.20      7.27   616.00     81.00%
  Latency Distribution
     50%    1.67ms
     75%    1.71ms
     90%    1.75ms
     99%    1.84ms
  17837 requests in 30.01s, 11.08GB read
Requests/sec:    594.42
Transfer/sec:    378.23MB 

nginx.exe

$ wrk -t1 -c1 -d30 http://172.16.220.205:80/xml.zip --latency
Running 30s test @ http://172.16.220.205:80/xml.zip
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.00ms  176.83us   4.04ms   69.41%
    Req/Sec   500.61     13.14   555.00     71.76%
  Latency Distribution
     50%    2.01ms
     75%    2.12ms
     90%    2.22ms
     99%    2.39ms
  14999 requests in 30.10s, 9.32GB read
Requests/sec:    498.31
Transfer/sec:    317.14MB 

Werkzeug + Flask

python.exe server.py -h 172.16.220.205 -g wz -f xml.zip -b

$ wrk -t1 -c1 -d30 http://172.16.220.205:5000/ --latency
Running 30s test @ http://172.16.220.205:5000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.46ms  523.74us   9.90ms   62.01%
    Req/Sec   274.58     26.55   343.00     76.00%
  Latency Distribution
     50%    3.62ms
     75%    3.87ms
     90%    4.03ms
     99%    4.30ms
  8204 requests in 30.00s, 5.10GB read
Requests/sec:    273.46
Transfer/sec:    174.04MB 

Waitress + Flask

python.exe server.py -h 172.16.220.205 -g wr -f xml.zip -b

$ wrk -t1 -c1 -d30 http://172.16.220.205:5000/ --latency
Running 30s test @ http://172.16.220.205:5000/
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.01ms  605.99us  11.31ms   67.94%
    Req/Sec   100.24      4.73   111.00     78.74%
  Latency Distribution
     50%   10.11ms
     75%   10.48ms
     90%   10.71ms
     99%   11.04ms
  3004 requests in 30.10s, 1.87GB read
Requests/sec:     99.80
Transfer/sec:     63.51MB 

[AnimaZer]

Hi, yesterday I conducted load testing of 2x wsgi, unfortunately, waitress defeated fastwsgi :)
I ran it together with nginx
fastwsgi:

waitress:

[AnimaZer]

fatwsgi could not withstand the high load

[AnimaZer]

I don't know where to write, so I wrote here))

[AnimaZer]

fastwsgi.pdf
waitress.pdf

[AnimaZer]

I hope this information will be useful.

[remittor]

@AnimaZer

I conducted load testing of 2x wsgi, unfortunately, waitress defeated fastwsgi :)

The waitress server, by default, initializes 4 worker threads, which process incoming requests.
Proof link: https://github.com/Pylons/waitress/blob/main/src/waitress/runner.py#L113

How many threads (processes) were used for the FastWSGI server?

[AnimaZer]

apparently I was running in 1 thread, how can I run in multiple threads?

[remittor]

Under Windows, it can now only be used in single-process mode (1 thread).
Addition info: https://github.com/libuv/libuv/blob/244df24bf411a396ceaf69f8a80a98e5629ee584/src/win/tcp.c#L271-L279

Therefore, for comparison, it is worth waitress making the use of 1 thread.