Measuring response times (routing times) for each framework (middleware). Each framework has to have two features; routing and parsing path parameters.
Last update: 2018-01-18
OS: Darwin (version: 17.3.0, arch: x86_64)
CPU Cores: 8
- iron (rust)
- router_cr (crystal)
- nickel (rust)
- raze (crystal)
- japronto (python)
- fasthttprouter (go)
- kemal (crystal)
- rocket (rust)
- iris (go)
- echo (go)
- aspnetcore (csharp)
- gorilla_mux (go)
- plug (elixir)
- gin (go)
- phoenix (elixir)
- vapor (swift)
- perfect (swift)
- sanic (python)
- kitura (swift)
- clusterexpress (node)
- akkahttp (scala)
- express (node)
- roda (ruby)
- jester (nim)
- criollo (objc)
- sinatra (ruby)
- rails (ruby)
- rust (iron)
- crystal (router_cr)
- python (japronto)
- go (fasthttprouter)
- csharp (aspnetcore)
- elixir (plug)
- swift (vapor)
- node (clusterexpress)
- scala (akkahttp)
- ruby (roda)
- nim (jester)
- objc (criollo)
Language (Runtime) | Framework (Middleware) | Max [sec] | Min [sec] | Ave [sec] |
---|---|---|---|---|
ruby | rails | 169.990758 | 169.245815 | 169.500696 |
ruby | sinatra | 48.785529 | 48.654910 | 48.711666 |
ruby | roda | 18.727357 | 18.608974 | 18.661612 |
crystal | kemal | 4.178043 | 4.054511 | 4.129273 |
crystal | router_cr | 3.455170 | 3.376436 | 3.427589 |
crystal | raze | 3.562387 | 3.444175 | 3.536099 |
go | echo | 5.256643 | 4.904162 | 5.048713 |
go | gorilla_mux | 5.390430 | 4.980591 | 5.142020 |
go | iris | 5.244560 | 4.795147 | 4.989660 |
go | fasthttprouter | 3.715726 | 3.595522 | 3.662270 |
go | gin | 5.682742 | 5.357033 | 5.502924 |
rust | iron | 3.485740 | 3.357875 | 3.412701 |
rust | nickel | 3.485643 | 3.416737 | 3.464607 |
rust | rocket | 4.260527 | 4.139049 | 4.211128 |
node | express | 13.895781 | 13.413687 | 13.619828 |
node | clusterexpress | 9.048196 | 7.710249 | 8.209995 |
elixir | plug | 5.599585 | 5.259777 | 5.456209 |
elixir | phoenix | 5.693897 | 5.393461 | 5.618488 |
swift | vapor | 6.296311 | 5.985685 | 6.120673 |
swift | perfect | 6.570743 | 6.186247 | 6.362126 |
swift | kitura | 7.643707 | 7.318455 | 7.467837 |
scala | akkahttp | 9.675075 | 7.849877 | 8.536331 |
csharp | aspnetcore | 5.405356 | 4.861280 | 5.128910 |
python | sanic | 7.291824 | 6.076758 | 6.445577 |
python | japronto | 3.587456 | 3.509572 | 3.544309 |
nim | jester | 27.304759 | 27.177061 | 27.240023 |
objc | criollo | 28.879014 | 27.740031 | 28.389459 |
- Ruby
- Crystal
- Go
- Rust
- node
- Elixir
- Swift
- Scala
- C#
- Python
- Nim
- Objective-C
See Development section when you want to add new languages or frameworks.
We want to know the response time (routing time), not a usability. So full-stack framework is at a disadvantage.
- Each server has no special logics.
- Each server's executable is named as
server_[Lauguage]_[Framework]
. (For example,server_ruby_sinatra
) - There are only 3 routes
- GET '/' return status code 200 with empty body
- GET '/user/:id' return status code 200 with the id
- POST '/user' return status code 200 with empty body
Required environment -> See Current target frameworks(middlewares)
Neph is a modern command line job processor that can be substitute for make
command.
To compile servers and benchmarker,
> neph
For each language,
> neph ruby
For each framework,
> neph rails
See neph.yaml
To compile servers and benchmarker,
> make
For each language,
> make ruby
For each framework,
> make rails
You can take a benchmark by
> bin/benchmarker
For each language
> bin/benchmarker ruby
For each framework
> bin/benchmarker rails
For comparison (Comparing rails, kemal and router.cr in this example)
> bin/benchmarker rails crystal
If you take it manually, you can run each server by
> bin/server_[Language]_[Framework]
and run client by
> time bin/client
You can set # of threads and # of the loops of the request(there are 3 requests in a loop) by
> time bin/client -t 16 -r 1000
In the above example, 16 threads requests 1000 * 3 times. So 48000 requests are sent in total.
Setup servers by using docker
is under WIP. Currently, crystal and ruby servers are supported. For example
docker-compose up rails
Then you can run your client by
time ./bin/client
- Give me PR when you want to add other web frameworks
- Give me PR when you can tuning each framework (under the rule)
/[language]/[framework]/[codes]
<- Project itselfbenchmarker/benchmarker.cr
<- Adding it as a target toREADME.md
<- Adding it as a target framework of the listMakefile
neph.yaml
(optional)
Anyway, you don't have to care about details since maintainer can fix them after merging it. The result will be updated by maintainer.
- Fork it (https://github.com/tbrand/which_is_the_fastest/fork)
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am 'Add some feature')
- Push to the branch (git push origin my-new-feature)
- Create a new Pull Request
- tbrand Taichiro Suzuki - creator, maintainer
- OvermindDL1 OvermindDL1 - maintainer
1AE9P6TUVik1rJGQhaSqGWRk1oAQ3DJnmo