Round 22 Completed about frameworkbenchmarks HOT 28 OPEN

No Bun or Deno in this one? I think these might be the two most wanted benchmarks this year.

from frameworkbenchmarks.

Round 22 has been posted. Blog post to follow early next week. Thanks everyone for your contributions!

from frameworkbenchmarks.

Results look good. We will be posting them this week. Still looking for some write ups!

from frameworkbenchmarks.

I have written a blog post about TFB and object pascal - yes, we added our object pascal framework in round 22!
https://blog.synopse.info/?post/2023/10/31/Pascal-in-the-race%3A-TFB-Challenge-Benchmarks

And in fact, next week I will go to Germany for two conferences at EKON 27, speaking about TFB and comparing good old object pascal solutions to the most trendy using Rust, Go or JS. https://entwickler-konferenz.de/en/

Here is a paragraph about what we did on our mORMot framework in respect to TFB, through a lot of iterations:

About how we maximize our results for TFB, we tried several ways of accessing the DB (ORM, blocking, async), reduced the syscalls as much as possible, minimized multi-thread locks especially during memory access (a /plaintext request requires no memory allocation), and made a lot of profiling and micro-optimizations. The benefit of the object pascal language is obvious: it is at the same time a high-level language (with interfaces, classes and safe ARC/COW strings), safe and readable, but also a system-level language (with raw pointers and direct memory buffers access). So the user can write its business code with high level of abstraction and safety, but the framework core could be tuned to the assembly level, to leverage the hardware it runs on. Finally, OpenSource helped a lot having realistic feedback from production, even if the project and the associated FreePascal compiler are maintained by a small number of developers, and object pascal as a language is often underestimated.

from frameworkbenchmarks.

I'll be out of the office for a couple of days this week, so hoping to get the blog post and results up on the site by the end of the week or early next week at the latest; so last call for any other folks looking to get a blurb in the blog post.

from frameworkbenchmarks.

The JavaScript frameworks I checked are still using the old NodeJS version 18.
Now that NodeJS 20 has LTS status, this should be used as the new default, especially because version 20 got really great performance improvements.

from frameworkbenchmarks.

There are both Bun and Deno here - but they only implement plaintext and json

from frameworkbenchmarks.

@nbrady-techempower I missed this :( I still have time to send a wrap-up for me (adding @vietj in cc too) re Netty/Vertx/Quarkus?

from frameworkbenchmarks.

List a few of Rust's performance optimizations.

In a real production environment, several approaches can be tried to optimize the application:

1. Specify **memory allocators**
   
   * [`mimalloc`](https://crates.io/crates/mimalloc)
   * [`snmalloc-rs`](https://crates.io/crates/snmalloc-rs)
   * [`jemallocator`](https://crates.io/crates/tikv-jemallocator)

2. Declaring **static variables**
   
   * [`once_cell`](https://crates.io/crates/once_cell)
   * [`lazy_static`](https://crates.io/crates/lazy_static)

3. Putting a small portion of data on the **stack**
   
   * [`smallvec`](https://crates.io/crates/smallvec)
   * [`tinyvec`](https://crates.io/crates/tinyvec)

4. Using a `capacity` to new **vector** or **hash**, a least capacity elements without reallocating

5. SIMD

Most of the stuff mentioned are either micro optimization or even can be harmful(in certain cases).

1. Custom memory allocator does not impact perf in real world. You use it when you need extra features(security, debug/profiling capabilities and customizable behaviors etc).
2. Global variable has it's pros and cons and you should only use it when needed for reason other than micro optimization.
3. Inline small data is micro optimization and it has limitation where you must be able to predict the data is moving around the inline water mark. Otherwise you may end up with a strictly inferior data structure.
4. Reducing memory reallocation is mostly about reducing fragmentation and perf impact is on micro level. Over allocating upfront can possibly result in high memory usage so as point 3 whether you can predict the data size is the key.
5. In real world web app you better just choose the Rust crate that can do simd for you rather than optimize it yourself.

In general you should not take anything from tfb benchmark and simply consider it useful in real world. Context and use case determines how you optimize your code.

btw: xitca-web (bench code not including dependencies) does not do 2,3,5 and still remains competitive in micro bench can be used as a reference.

from frameworkbenchmarks.

@hiqsociety The web results are for round 22. To watch continuous runs happen, that's on https://tfb-status.techempower.com

from frameworkbenchmarks.

Where is Drogon framework ? you should apply framework with best performance in last rounds, in new benchmark, too.

from frameworkbenchmarks.

@franz1981 Yes, I got yours. Thanks for sending that over!

from frameworkbenchmarks.

@franz1981 Ok, got it for real this time. Thanks!

from frameworkbenchmarks.

List a few of Rust's performance optimizations.

In a real production environment, several approaches can be tried to optimize the application:

1. Specify memory allocators

   • mimalloc

   • snmalloc-rs

   • jemallocator

2. Declaring static variables

   • once_cell

   • lazy_static

3. Putting a small portion of data on the stack

   • smallvec

   • tinyvec

4. Using a capacity to new vector or hash, a least capacity elements without reallocating

5. SIMD

from frameworkbenchmarks.

@nbrady-techempower thanks, I've sent an email with the changes

from frameworkbenchmarks.

There are both Bun and Deno here - but they only implement plaintext and json

ah, I see, there's a separate TypeScript folder.

how come? I mean, TS doesn't have a runtime - so in terms of performance, it's going to be dependent on whatever the emitted JS is running on. Any framework that supports TS also supports JS, and most of the frameworks in the JavaScript folder also support TS, so it doesn't seem very meaningful to break TS into a separate category from JS, I think? it's JavaScript, with types.

from frameworkbenchmarks.

@mindplay-dk

it doesn't seem very meaningful to break TS into a separate category from JS, I think? it's JavaScript, with types.

The separation isn't meant to mean anything other than the programming language the code is written in; not what it compiles/transpiles to.

@Robbson you are welcome to upgrade any languages and frameworks that you think are out of date. Even better if you can ping the maintainers while you do it.

from frameworkbenchmarks.

Thanks @nbrady-techempower for posting Round 22! Looking forward to the blog post.

from frameworkbenchmarks.

@nbrady-techempower Could you also add releases/tags for round 21 and 22 to this repository, please?

from frameworkbenchmarks.

How to participate in this testing competition
https://github.com/hggq/paozhu
C++ Web framework

from frameworkbenchmarks.

@volyrique Ah, thanks for calling that out.

@franz1981 I think the email I thought was from you was from someone else. I'm about to get this blog posted. Would you mind resending to [email protected]

from frameworkbenchmarks.

@nbrady-techempower I'm [email protected] :) it should be (I'm Francesco, as first name)

from frameworkbenchmarks.

@nbrady-techempower possible to update the website on the latest pull request? i've submitted mine but was not updated.

golang's sprapp

from frameworkbenchmarks.

@NateBrady23 Perhaps it is time to remove the message Round 22 official run has started... from the results dashboard 😉?

from frameworkbenchmarks.

Hi, when will the blog post be published?

from frameworkbenchmarks.

Hi, what is the precise version information of MySQL，MongoDB & Postgres？I can not find on the site or github.

from frameworkbenchmarks.

@firebank How precise would you like that information to be? You could look at the database Dockerfiles for a fairly accurate idea:

• MongoDB
• MySQL
• PostgreSQL

from frameworkbenchmarks.

Drogon didn't go anywhere, but it failed to build during the benchmark run for round 22.

from frameworkbenchmarks.