kylemayes / microbench Goto Github PK

The R^2 values from this crate look unrealistically high - way higher than I would expect based on my experience with variance in benchmarks.

I skimmed through lib.rs and didn't see "log" or "ln" so I think that may be the issue - linear regression assumes homoscedasticity and that assumption is violated by using a geometrically distributed number of iterations.

• https://en.wikipedia.org/wiki/Linear_regression#Assumptions
• https://en.wikipedia.org/wiki/Homoscedasticity

You should be able to fix it by using log-log regression - take the log of both iterations and elapsed time before computing the linear regression.

My code has particularly long labels. So, despite the attempt to make everything line up nicely in bench_impl, it doesn't. If we could use custom format strings, I could just make the fields wider.