Reserve cpus for benchmarking about hyperfine HOT 2 OPEN

This is a great question. I don't have the answer. I have experimented with hyperfine + cpuset in the past and I believe that it may have helped reducing outliers, but I can't provide definitive proof.

Note that context switches are not the only source of noise. There's also I/O, which often plays a huge role when benchmarking CLI apps, but that obviously spends a lot on your use case. cpuset will not help with this.

from hyperfine.

Ok, so my understanding is that if someone sees notable outliers/variability in their results, they should consider running hyperfine with cpuset as an option, but remember that this would help only if the variability is due to context switches.

from hyperfine.