Giter Club home page Giter Club logo

Comments (2)

sandersaares avatar sandersaares commented on June 8, 2024 1

Increasing concurrency to 1024 after fiddling with config files shows:

  • The benchmark itself becomes unstable (some verifications pass, some fail, randomly)
  • Absolute numbers increase (even at low core counts!)
  • Scaling curve remains very similar, peaking around 20-25 cores

Not sure if it proves or disproves anything specific but it definitely underlines that some upgrades are needed to ensure the benchmark results are believable at high core counts.

image

from frameworkbenchmarks.

joanhey avatar joanhey commented on June 8, 2024

Exist other issues like this one (#5626 is an example).

We need to remember that this bench is run with that enterprise servers, but also we run it in the cloud.
And specially we need that run locally with only one computer in local for test, before we send it the PRs to this repo.

I understand the issue, but we need to remember that this benchmark is 11 years old, and that is changing with the time. The first bench was in 2013 with i7-2600k (2ΒΊ generation) with 8Gb and in the cloud with EC2.
And still all results are important in any environment, and that normally the results are relative comparable using any servers. Also not everybody will have enterprise servers in production for their applications.

I like the idea from @an-tao #5626 (comment)

Do we really need 16 and 32 concurrent users actually, even testing in local, for json, db and fortunes tests. I think that not. And the best think is that we will have faster runs, as any test need ~20 sec extra with any framework.
And I really like to see the data table information, is very informative, but the majority of people never watch it.
Still 16 and 32 concurrency give very little information actually.

1024 in place of 16 and 32 seems Ok now. But for more, it'll need a bigger toolset change. Auto calculate (checking the servers resources) or manually have 3 configs for: local, faster and enterprise.

@NateBrady23 do you think that are still relevant 16 and 32 concurrent connections ? We will have faster runs without them.

from frameworkbenchmarks.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.