python-web-perf's Introduction

Python webserver performance comparison

Heavily based on Cal Paterson's benchmark, with some bugs fixed and some new web servers and web frameworks added.

A description of this benchmark and some results are published in my article Ignore All Web Performance Benchmarks, Including This One.

Benchmark code

You can find the implementations of this benchmark for the different web frameworks, as well as the startup commands for all web servers, in the src directory.

Running the benchmark

The run-docker.sh command creates containers for nginx, pgbouncer and postgres and sets them up for the test. Then it builds a local image with all the application code and server startup scripts. It finally runs each of the tests in random order by starting a web server container and a load generator container. To run one or more specific tests you can pass the test name(s) as arguments. The list of available tests is in the file tests.txt.

Look at the top of run-docker.sh for the list of environment variables that configure the test environment.

python-web-perf's People

Stargazers

Watchers

python-web-perf's Issues

high number of failed_reqs, psycopg2.OperationalError: could not connect to server

Hi,
I tried running the benchmark on ubuntu 18.04, with Intel® Core™ i7-8850H CPU @ 2.60GHz × 12 and I have an issue with db connections.

NUM_CLIENTS=100
NUM_CONNECTIONS=20000
NUM_WORKERS_SYNC=12
NUM_WORKERS_ASYNC=6
NUM_DB_SESSIONS=100
DB_SLEEP=0.02

First run : gunicorn 20.0.4 with meinheld 1.0.2 -> gunicorn 20.0.4 -> gunicorn 20.0.4 with gevent 20.9.0

framework	server	workers	reqs/sec	P50	P99	db_sessions	total_reqs	failed_reqs	cpu_%	ram_mb	net_in_mb	net_out_mb	pids
Flask 1.1.2	gunicorn 20.0.4 with meinheld 1.0.2	6	2084.08	41	107	100	20000	0	555.75	130.2331392	16.6	14.3	8
Flask 1.1.2	gunicorn 20.0.4 with gevent 20.9.0	6	1588.33	53	190	100	20000	0	470.05	156.13296640000002	52.9	45.7	8
Flask 1.1.2	gunicorn 20.0.4	12	540.34	120	348	12	20000	11566	971.2	232.3644416	32.2	27.8	14

Second run : gunicorn 20.0.4 with meinheld 1.0.2 -> gunicorn 20.0.4 with gevent 20.9.0 -> gunicorn 20.0.4

framework	server	workers	reqs/sec	P50	P99	db_sessions	total_reqs	failed_reqs	cpu_%	ram_mb	net_in_mb	net_out_mb	pids
Flask 1.1.2	gunicorn 20.0.4 with meinheld 1.0.2	6	2078.64	41	106	100	20000	0	551.05	130.1282816	16.5	14.3	8
Flask 1.1.2	gunicorn 20.0.4 with gevent 20.9.0	6	921.52	87	389	100	20000	11662	582.59	157.810688	32.1	27.8	8
Flask 1.1.2	gunicorn 20.0.4	12	539.52	200	283	12	20000	7879	761.26	231.735296	45.6	39.5	14

For gunicorn 20.0.4, I saw a lot of failure at the beginning of the run but not at the end.

And the exception raised inside the perf-app container.

[2020-11-30 15:14:39,832] ERROR in app: Exception on /test [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/app_flask.py", line 10, in test
    a, b = get_row()
  File "/app/sync_db.py", line 17, in get_row
    conn = psycopg2.connect(dbname='test', user='test', password='test',
  File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Cannot assign requested address
        Is the server running on host "perf-dbpool" (172.21.0.4) and accepting
        TCP/IP connections on port 5432?

➜  ~ docker run --rm busybox cat /proc/sys/net/ipv4/ip_local_port_range
32768	60999
➜  ~ docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:06 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Recommend Projects

miguelgrinberg / python-web-perf Goto Github PK

python-web-perf's Introduction

Python webserver performance comparison

Benchmark code

Running the benchmark

python-web-perf's People

Stargazers

Watchers

Forkers

python-web-perf's Issues

high number of failed_reqs, psycopg2.OperationalError: could not connect to server

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent