Add possibility to ignore setup phase about hyperfine HOT 4 CLOSED

Thank you for the feedback.

Is it possible to split-up your script into two parts (generate-keys.sh, benchmark.sh)? In this case, you can use the --prepare option of hyperfine:

> hyperfine --prepare ./generate-keys.sh ./benchmark.sh

from hyperfine.

I thought about that, yes, but I'd still measure the time required to read the keys from disk then. I'm running benchmarks for 1, 10, 100, 100 keys and 1, 10, 100, 1000 messages currently and simply subtract the time required for 1 messages from the final value, which gets me just the time required for 999 message validations without the setup phase.

from hyperfine.

I see. Making the setup phase negligible by increasing the number of iterations is certainly a good idea if you are only interested in the time for a single iteration.

However, I'm afraid that hyperfine is not really the best tool for your use case. It is really about quick, whole-program benchmarks. If you need finer granularity, I think you would be better off by using a benchmarking library from the programming language that you write your script in (or is it a shell script? 😄).

Waiting for a magic string value in the output of the benchmarked program doesn't really sound like a very natural extension for me, to be honest. However, if you have any other ideas on how to make this work, I'm certainly open to hearing about them!

from hyperfine.

Closing this for now. Happy to re-open this, if there is more to discuss.

from hyperfine.