sqr_ family of benchmarks incorrect about fp-benchmarks-aachen HOT 2 CLOSED

While this is annoying, it is also kind of cool. What happens is that the test attempts to show that squaring and then taking the square root of a number results in either infinity (on overflow), zero (on underflow) or roughly the same number as before.

What I missed when writing the test is that the approximation can be quite significant, when not quite underflowing! The example input you mention results in the smallest denormal number after squaring, thus losing 22 bits of precision. This causes the final result to not be an over-/underflow situation, but still miss the expected range for the result.

I really like this result, as it perfectly shows the pitfalls of floating point numbers and how our tools can find even pretty damn exotic failures - we should probably discuss this in the paper!

from fp-benchmarks-aachen.

@danielschemmel Just to note we don't necessarily need to fix this benchmark during our runs. Originally your tool was penalised due to it not matching the spec but now that we incorporate information from attempting to replay all bug reporting KTest files your false positive gets turned into a true positive during ranking.

We should definitely fix the benchmark at some point though.

from fp-benchmarks-aachen.