Include ravendb benchmarks in dotnet/performance about ravendb HOT 3 OPEN

Yes, that would absolutely be fine by us.
Note that we are routinely updating our code to reflect new behaviors as changes to the runtime occurs

For example, we used to have our own lock free dictionary and switch to ConcurrentDictionary in some cases recently.

May be worth also re-visting that one in a while

from ravendb.

I have started taking a look to figure out which benchmarks would be the best candidates to copy over into the performance benchmark tests. I am wondering if there are any specific benchmarks that have been useful for catching regressions in the past or that y'all think would be best to include in the copied tests?

It seems that the Micro, ServerStoreTxMerger, Voron, and VxSort benchmark sets are the ones already using Benchmark DotNet, which is what we use for most of our tests already. As such, my initial plan is to focus on the Micro benchmarks as a starting point for the benchmarks to setup. Please let me know any thoughts!

from ravendb.

For catching platform regressions (that could happen when we update .Net version), I would say that from the Micro project you could include the following:

• HopeEncoder
• IntegerEncoder
• TypeCache
• PIRandomRead
• PIRandomWrite
• Compare
• Diff

BlittableJsonBench.cs has some that are important to track over time. The Voron ones are too low level (specially the Corax ones) but IF I would want to have a baseline I would include the BTREE and the TABLE ones. Corax ones are too micro to make a dent because we know that bottlenecks are not there anyways, you would be wasting compute power for something that would not change the outcome. VxSort are not important, they are too low level and just used for metrics on the underlying sorting algorithm for Corax.

BUT, only the ones that call the RavenDB code... in all cases most of the benchmarks are immortalized code (code that we used to have or had developed to test optimization ideas) in the project itself to use as comparison. We keep them also because the underlying optimizations performed at the platform level would throw a curve-ball or two between major releases.

It's important to know though that those are mostly internal calls, and therefore are not required to stay the same even within minor releases.

from ravendb.