Graph execution time as a function of the size of the input about benchee_html HOT 2 OPEN

👋

Sorry, I must have somehow missed this issue 😞

Sounds like a good idea. I'd like to just specify a "special" input that includes information about the size or a way to get the size. Specifying the size, seems to be hard because any kind of input data structure is a valid input for benchmarking inputs - who am I to judge?

The best think that I could come up with right now is an "input quantifier" function (name pending) that receives the inputs and spits out a number that would be used to compare and graph it. Like in your example it'd be like input_quantifier: fn file_name -> count_lines(file) end or something.

That's a big addition though - not just for graphing but we could even try to make a guess if a function is linear or not given 3 or more input data points. This is probably harder than I imagine right now though 😁

Definitely past 1.0 (which should land soon) - I'll keep it in mid though.

Thanks for the excellent idea 🎉

from benchee_html.

This seems like something that could work with the proper support for using property testing generators with Benchee. For any input that has a size we could generate inputs of increasing size and use that size to graph against the result of the benchmark.

from benchee_html.