klen / py-frameworks-bench Goto Github PK

do you have any plan to add this ?

Would be really helpful to add this web server

• https://github.com/squeaky-pl/japronto

and also if this benchmark coincide with your results

• https://medium.com/@squeaky_pl/million-requests-per-second-with-python-95c137af319

Thanks.

The benchmark is missing the number of requests/s. I ran a similar test as the JSON one, and yes flask is faster than aiohttp with those wrk settings. However, aiohttp handles more than 11 times more requests as flask. 4098 vs 359 in my case.

The benchmark doesn't include cherrypy

can you measure web2py too on your benchmarks?

As stated here http://flask.pocoo.org/docs/0.11/api/#flask.json.jsonify flask pre-processes the json response with "pretty print" if the header X-Requested-With: XMLHttpRequest was not used in the request so, the "encode a object to json" test requests should have that header to make things fair.

Several frameworks's apps are written in a manner that does not reflect the usage suggested in their docs.

For example, the FastAPI documentation suggests using File() to have files extracted from forms and injected: https://fastapi.tiangolo.com/tutorial/request-files/. But in the benchmarks the Request.form is used just like in Starlette. This is obviously going to be faster, but then it's not using any of the features the framework provides on top of Starlette. If that's what users were doing, they'd be using Starlette.

I suggest that the apps get restructured to better reflect the documented usage of each framework. That is much more useful to users (and framework developers) and would help highlight issues like tiangolo/fastapi#4187

The django database connection doesn't have CON_MAX_AGE set, which means that it will make a new database connection every time (the default behaviour), rather than keeping the TCP/IP connection open, which can be very slow.

Hi,

I'd like to fork the codebase to do some improvements and use it to verify some other behavior, but I noticed this project has no license.

can you add a license? If there's no strong preference, MIT would be simplest, but really any license that lets me use the code in a derivative work is sufficient.

Thanks for your time!

Since you're varying the ORM based on the http framework the benchmarks seem vary misleading. Some were using pee_wee, some sqlalchemy, and django was using django. Couldn't you make the benchmarks all use pee_wee so you're bench-marking the web delivery part of the framework in isolation of ORM performance and features?

Hi ! I would like to add Vibora to the benchmarks list. Are you open to a pull request ? :)

I wanted to point out the discussion at https://news.ycombinator.com/item?id=9724999 in case it wasn't on your radar.

just want to see the real performance of sanic~

Is empty response faster than json/remote/message?

maybe help.

How about adding Masonite framework in the list?
Masonite Framework

Starlite is gaining in popularity due to it's appealing featureset. Please include it.

Hi @klen 👋

I think the pipeline is not running anymore because GitHub limits it when a repository doesn't receive changes for a period of time. The last one was https://github.com/klen/py-frameworks-bench/actions/runs/1982085780 👀

Would you mind checking it? If you don't have time, @adriangb or myself can also help here. 🙏

May I suggest "test setup" or "methodology?"

Cheers.

Проще показать, внимание на скриншоты.

gunicorn -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8080 app:app — самый тормозной вариант, не понимаю что происходит, и не хочется разбираться, но на каждый запрос принтится лого, отсюда и результат.

gunicorn -k sanic.worker.GunicornWorker -b 0.0.0.0:8080 app:app — более адекватный вариант, опять же, разбираться лень, но принтится access_log, что влияет на результат, как отключить, я сходу не нашёл.

sanic --host 0.0.0.0 --port=8080 --workers=1 app:app — дефолтный и наиболее корректный способ запуска, дающий результат на уровне blacksheep.

И, собственно, blacksheep, для сравнения:

Please add robyn which uses rust under the hood.

https://github.com/sansyrox/robyn
https://robyn.tech/

Would like to see the performance of these two compared to Flask itself.
Quart: https://pgjones.gitlab.io/quart/
Klein: http://klein.readthedocs.io/en/latest/index.html

The ORM benchmark seems to state the 50% time for AIOHttp incorrectly - the value listed in the table is actually the min (as shown on the graph). The value for 50% is rather 902.72. Screenshot below for details:

I'm getting a TASK [bench.setup git]

which fails because it Failed to find required executable git.

I did a vagrant ssh and apt-get install git and then it passed, only to fail on make in TASK [bench.setup command].

T2 instances inherently rely on excess CPU cycles and therefore do not provide for a reliable performance baseline upon which benchmarks can be run accurately.

Considering running the benchmarks on a dedicated private instance.

Some framework use simplejson as default(flask), some use built-in json module(tornado).

Right now it comparing only requests, not websockets

It would be awesome if the Python's Web Framework Benchmarks were updated to reflect the current state of affairs (2018/2019).

Which framework is best for what? Could you give some hints to a manager (or six year old kid) please ;-)

If I didn't include the --provider=virtualbox vagrant tried to use libvirt, with this it went along.