Giter Club home page Giter Club logo

php-frameworks-bench's Introduction

PHP Frameworks Bench

Test PHP benchmark GitHub release (latest by date) GitHub

๐Ÿ“Š This project attempts to measure the minimum overhead (minimum bootstrap cost) of PHP frameworks in the real world.

So I think the minimum should not include:

  • cost of template engine
  • cost of database manipulation
  • cost of debugging information

Benchmarking on components like template engines or ORM/Database libraries is out of the scope of this project.

Benchmarks

Latest

Environment

For PHP 8.2 check out this link.

  • Ubuntu 22.04 LTS x86_64
    • PHP 8.3.0
    • OPCache Off
    • Apache 2.4.52
    • WRK 4.2.0 (5 min)
    • CPU Core i7-3770K/4.4Ghz
    • Memory 16G

Results (2023/12/1)

These are my benchmarks, not yours. I encourage you to run on your -production equivalent- environments.

Frameworks Benchmark Results Graph Throughput Frameworks Benchmark Results Graph Memory Frameworks Benchmark Results Graph Execution Time Frameworks Benchmark Results Graph Included Files

framework requests per second (rps) relative (rps) peak memory relative (mem)
pure-php 27,077.31 321.6 0.42 1.0
kumbia-1.1 5,921.29 70.3 0.54 1.3
fastroute-1.3 4,516.12 53.6 0.56 1.3
phroute-2.2 4,198.83 49.9 0.58 1.4
leaf-3.5 1,487.36 17.7 1.11 2.6
fatfree-3.8 1,453.82 17.3 1.72 4.1
slim-4.12 798.17 9.5 1.57 3.7
ubiquity-2.4.x.dev 705.12 8.4 1.64 3.9
silex-2.3 543.66 6.5 2.20 5.2
yii-2.0-basic 503.76 6.0 2.56 6.1
fuelphp-1.9 429.18 5.1 2.61 6.2
lumen-10.0 325.27 3.9 3.61 8.6
symfony-5.4 309.74 3.7 3.77 9.0
codeigniter-4.4 302.21 3.6 3.62 8.6
symfony-7.0 286.46 3.4 4.07 9.7
symfony-6.4 280.83 3.3 4.11 9.8
cakephp-4.5 246.72 2.9 4.63 11.0
cakephp-5.0 243.96 2.9 4.74 11.3
laravel-10.2 84.19 1.0 13.06 31.0

OPCache On

Check out the video for more information.

Results with OPCache On/Off & How to add your framework

How to Benchmark

If you want to have benchmarks on PHP extension frameworks like Phalcon, you need to install the extension first, based on its own documentation.

1- Download & Setup:

# make sure to put it in the root of your web path
$ git clone https://github.com/myaaghubi/PHP-Frameworks-Bench.git

$ cd PHP-Frameworks-Bench

# optional
$ nano config

# run the setup & follow the progress
$ yes | bash setup.sh

2- Make sure everything is ok:

$ bash check.sh
# bash check.sh -t pure-php
# โœ” pure-php

3- Run benchmarks:

# bash benchmark.sh --help
$ bash benchmark.sh

4- Check the results:

Docker

Results with docker may or may not be reliable but in a situation you can use it like:

1- Change the base in config on the right port(8080 considered):

base="http://127.0.0.1:8080/PHP-Frameworks-Bench"
...

2- Run/download the docker:

$ bash docker-apache.sh

3- Run the benchmark:

# run it in another terminal
$ bash benchmark.sh

Commands

# use bash benchmark.sh --help
$ bash benchmark.sh -f -rapache -t pure-php

# run composer update for frameworks
$ bash update.sh

# to clean them all
# trouble? run it then
# you can use `bash setup.sh` again
$ bash clean.sh

# clear the cache of frameworks
$ bash clear-cache.sh

# show the results table
$ bash results.sh

To specify frameworks, put them with -t ... after each command:

# supported for `setup.sh`, `benchmark.sh`, `update.sh`, `clean.sh`, and `clear-cache.sh`
# bash benchmarks.sh --help
$ bash benchmark.sh -t laravel-10.2/ slim-4.12/ ...
...

Add Your Framework

Check out the Benchmarking Policy, to get more information watch the video OPCache On/Off.

Benchmarking Policy

  • Use composer to install frameworks according to their official documentation.
  • Use the default configuration.
    • Minimum changes on frameworks to have benchmarks.
    • Don't remove any components/configurations even if there is no use for them.
  • Include a controller class to get the Hello World! for each, based on the default template of each framework for controllers.
  • Turn off the debug mode and set the environment to production mode.
  • General optimization for the production environment, like --optimize-autoloader for the composer.

Some frameworks are optimized more than others, so some developers may think using default configuration is not fair. The dept of optimizing a framework depends on the skills/experiences of the developer too, so it's the rabbit hole and nonsense for ranking. Please note optimizing for the Hello World is not acceptable! Building the fastest Hello World application is not the goal of this project. I think the default configuration of frameworks is a good starting point to have a ranking.

If you find something wrong in my code, feel free to send a PR.

๐Ÿ” Donate

Don't forget to donate if you find it useful โ˜• ๐Ÿบ ๐Ÿธ ๐Ÿ”

ETH: 0x0ADd51D6855d2DF11BB5F331A3fa345c67a863b2

Ethereum

References

Note: This project is based on php-framework-benchmark, thanks to Kenjis. It is very old and abandoned, so I decided to split it from the origin and change/update it separately.

๐Ÿ™ Thanks to contributors:

๐Ÿ“„ For frameworks, I considered the official repos:

License

You are allowed to use this plugin/project under the terms of the GNU General Public License version 2.

Copyright (C) 2024 Mohammad Yaaghubi

php-frameworks-bench's People

Contributors

joanhey avatar myaaghubi avatar stdex avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

php-frameworks-bench's Issues

Only use the major version in the frameworks dir name

The frameworks create new versions very fast. We can use minor version and lock the version in composer, but it'll be a never ending change list.

I suggest to only use the major version in dir name.
If we want to know the exact version in the actual benchmark run, we can use composer show.

Actual diff in dir name and actual versions used in the bench.

Framework Dir name Actual version
Codeigniter 4.3 4.4.0
Fatfree 3.8.1 3.8.2
Laminas 2.0 2.2.0
Laravel 10.0 10.2.6
Leaf 3.3 3.4.1
Slim 4.11 4.12.0
Ubiquity 2.4.x.dev 2.5.2

For that list, we used:
https://github.com/myaaghubi/PHP-Frameworks-Bench/actions/runs/5984291041/job/16235305243#step:6:2

Please show the results with OPCache On

Nobody use an app or framework in PHP without OPCache, and less in production.

Even the shared hostings use it, and the Kenjis bench data also.
And it's a bad image for PHP, because a lot of people think that is slow, and without OPCache the numbers are not reallistic.

The `output_data.php` don't work with PHP-FPM in some frameworks

PHP-FPM has become the de-facto process manager for PHP, providing more flexibility than the venerable PHP module for Apache.

All frameworks that use the Symfony kernel, output no data after the Hello World!.

Internally, the HttpKernel makes use of the fastcgi_finish_request PHP function.

https://symfony.com/doc/current/components/http_kernel.html#component-http-kernel-kernel-terminate

Frameworks that fail:

  • Symfony 5.4 and 6.3
  • Laravel 10
  • Lumen 10
  • Silex

image

We need to search a solution.

Some microframeworks hardcode the URL prefix

Like say in the README: Used the default configuration

So before hand no framework know the URL, but some microframeworks have hardcoded the prefix for this exact benchmark.

Frameworks with this advantage:

  • Php-pure
  • Phroute
  • Fastroute
  • Slim4
  • FrameworkX

The rest need to calculate the prefix in each request.

Wrong results?

On my configuration:

Intel(R) Core(TM) i5-4670 CPU @ 3.40GHz
Ubuntu 20.04.3 LTS, php 8.1
|framework          |requests per second|relative|peak memory|relative|
|-------------------|------------------:|-------:|----------:|-------:|
|fatfree-3.7.3      |           9,224.12| 9,224.1|       0.39|     0.4|

|framework          |requests per second|relative|peak memory|relative|
|-------------------|------------------:|-------:|----------:|-------:|
|slim-4.9           |           6,940.34| 6,940.3|       0.38|     0.4|

~35% diff

In your results table ~300% diff

Create datetime dirs in the output [enhancement]

Like I always say, a benchmark is NOT a competition, but a very good tool to optimize the code.

If we create dirs with the datetime in the output, we can have later another graph with the performance over time.

We only need : for each dir in output, give me results.hello_world.log and display over time in a graph per framework.

And we can delete any dir in output, at any moment.
So a developer can measure the performance of the changed code over time.
And check the changes that make slower the framework between that time frame.
Also check visually the performance over different versions. Is it v 3.4.1 slower than 3.4.0 ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.