Comments (2)
Increasing concurrency to 1024 after fiddling with config files shows:
- The benchmark itself becomes unstable (some verifications pass, some fail, randomly)
- Absolute numbers increase (even at low core counts!)
- Scaling curve remains very similar, peaking around 20-25 cores
Not sure if it proves or disproves anything specific but it definitely underlines that some upgrades are needed to ensure the benchmark results are believable at high core counts.
from frameworkbenchmarks.
Exist other issues like this one (#5626 is an example).
We need to remember that this bench is run with that enterprise servers, but also we run it in the cloud.
And specially we need that run locally with only one computer in local for test, before we send it the PRs to this repo.
I understand the issue, but we need to remember that this benchmark is 11 years old, and that is changing with the time. The first bench was in 2013 with i7-2600k (2ΒΊ generation) with 8Gb and in the cloud with EC2.
And still all results are important in any environment, and that normally the results are relative comparable using any servers. Also not everybody will have enterprise servers in production for their applications.
I like the idea from @an-tao #5626 (comment)
Do we really need 16 and 32 concurrent users actually, even testing in local, for json, db and fortunes tests. I think that not. And the best think is that we will have faster runs, as any test need ~20 sec extra with any framework.
And I really like to see the data table information, is very informative, but the majority of people never watch it.
Still 16 and 32 concurrency give very little information actually.
1024 in place of 16 and 32 seems Ok now. But for more, it'll need a bigger toolset change. Auto calculate (checking the servers resources) or manually have 3 configs for: local, faster and enterprise.
@NateBrady23 do you think that are still relevant 16 and 32 concurrent connections ? We will have faster runs without them.
from frameworkbenchmarks.
Related Issues (20)
- New execution mode "profiling" HOT 6
- Enhancement request: disable pg_stat_statements when running anything but validation
- PHP 8.3 update [info]
- Inconsistent composite score best score computation HOT 2
- Holiday Break HOT 8
- Where to find the exact code that was used for Round22? HOT 1
- Expired SSL `tfb-status.techempower.com` HOT 1
- Actix failing build HOT 9
- New Server Set up HOT 57
- Most of the best-performing frameworks don't survive temporary db connectivity loss HOT 25
- [IMPORTANT] To further test fairness. Please consider this for next benchmark tests. HOT 1
- gRPC Framework benchmark HOT 1
- Please upgrade hardware for extreme testings HOT 1
- Randomize Framework order in runs HOT 11
- Add GraalVM native build for Quarkus HOT 3
- Add container name in tfb for local metrics HOT 1
- Create benchmarks using seastar HOT 1
- [Python] mrhttp: a project with many problems HOT 2
- What is the policy for removing unmaintained frameworks? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from frameworkbenchmarks.