realtimechris / json-performance Goto Github PK

Thanks so much for these JSON tests and for pushing for more performant JSON handling! I've been using these tests to help optimize the code.

Anyway, I fixed the recent issue Glaze introduced with handling your Discord test, so you could reimplement it now.

Typically, large JSON data files are minified. Glaze is optimized for reading minified data for this reason.

It's great that Jsonifier is fast at reading prettified files, but it would probably be best to compare both prettified and non-prettified parsing.

As per the title, the plots don't seem to get updated with the numbers and therefore don't reflect the actual numbers.

Prettified write performance in the benchmarks doesn't make sense to me, because it doesn't actually seem to be writing prettified data.

Glaze has built in support to write prettified data, but I don't think Jsonifier can. Jsonifier could call its prettify method after calling write, but this would be two steps and probably much slower.

Your custom vector has a lot of performance problems when it comes to implementation (I could provide specifics, but there are reserves lacking, not using noexcept move constructible, etc.). By not using std::vector you are introducing performance issues into the benchmark that cloud results.

You could always use custom allocators with std::vector if you want specific alignment.

The primary issue of not using std::vector (and an argument for using the built in allocators), is that this is the type 90% or more of other C++ libraries are using. So, you are testing code that is incompatible with the majority of other C++ code, and would require copies of data to make it compatible, which would kill performance.

You probably want the tests reflective of a normal C++ experience and not of your own vector implementation.

I've noticed you've been making a lot of values null, and this appears to make Jsonifier faster. But, Glaze is often actually running faster even though the benchmarks show Jsonifier as higher MB/s. This is because Glaze by default doesn't write out null members, so it is outperforming Jsonifier, but your benchmarks only consider the data rate versus time. If I just added a bunch of whitespace to Glaze output then your benchmarks would show Glaze as much faster. This makes the benchmark pretty meaningless.

Note that if you included the runtime with your results you would probably have caught this issue.

If null values are important, then Glaze's option skip_null_members should be set to false to compare with Jsonifier.

Additionally, when writing prettified data Jsonifier writes 4 spaces versus Glaze writing 3 spaces for the indent size. This means Jsonifier writes a lot more whitespace, which artificially skews the results (this includes the prettify test). This is why removing data from tests makes Jsonifier appear faster, because there is just more whitespace processing happening for Jsonifier. The benchmarks should just use the same number of spaces for the indentation width for a proper comparison.

Like Jsonifier, Glaze now has a compile time option for when the read input is known to be minified, improving performance. Just add .minified = true to the read options.

Example: glz::read<glz::opts{.minified = true}>(obj, buffer);

Thanks for the motivation to add this feature.