Raw binary support about glaze HOT 10 CLOSED

Yeah, I understood like 40% of it, and thats being generous lol. I am fairly new to advance C++, right now I wrote what you are writing for my work but by using https://github.com/veselink1/refl-cpp (we do a lot of network things, and are bound by ethernet's 1500bytes limit, so tags are a no no). Excited to delete all that macro code and move to glaze entirely!

Do let me know if you need any specific help.

I'm trying to figure out what to name these functions for writing/reading untagged binary.

read_binary_untagged and write_binary_untagged is a lot better than flat, since flat in my mind means removing depth, I know thats how the binary data will be saved, but untagged is better name I think.

from glaze.

After giving this some more thought, I've realized that really the only thing we would want different than a normal BEVE implementation is to drop tags (keys) for structs. If we use BEVE arrays for structs then we reduce memory consumption and have an excellent binary format that would match most raw implementations, and we don't need another specification. We can still be 100% BEVE conformant and just not write out the keys.

All we need is a compile time option in glz::opts such as write_structs_as_arrays. This can also be applied to JSON. But, now binary will work across compilers and operating systems, and has all the benefits of using BEVE without paying for tags.

from glaze.

That sounds great! How far is this implementation? If you can guide me, I can work on a PR?

from glaze.

@kalradivyanshu, thanks for offering to make a PR. In glz::opts a boolean write_structs_as_arrays needs to be added. We then need structs that satisfy the concepts of reflectable or glaze_object_t to behave similarly to how std::tuple is serialized. A minor challenge is that the concepts as not associated with glz::opts, so I'm not sure the best way of avoiding code duplication. For reflectable types we can call to_tuple and then serialize this tuple, that should be really straightforward. For glaze_object_t we'll probably need a bit more code. If you want to try to accomplish this and submit a pull request, go for it. I should also be able to work on this soon, and it should't be too much of an effort.

from glaze.

I'm working on this now, so I'll let you know if I can use some help, and I'll push a branch soon.

from glaze.

I'm trying to figure out what to name these functions for writing/reading untagged binary.
read_binary_untagged?
read_binary_flat?

from glaze.

I'll add the note that glaze already has a glz::array, so using this the structs_as_arrays option with glz::object is unnecessary, but I think I'll support it anyway. glz::array should compile a bit faster. But, by supporting this option with binary it allows JSON to include keys and binary to write out without keys.

from glaze.

I've merged in #671, which adds the option structs_as_arrays and adds the helper functions read_binary_untagged and write_binary_untagged. Unit tests have been included as well.

@kalradivyanshu, let me know if you run into any issues with this new feature.

from glaze.

Woah! You move fast, will check it out!

from glaze.

I've merged in #671, which adds the option structs_as_arrays and adds the helper functions read_binary_untagged and write_binary_untagged. Unit tests have been included as well.

@kalradivyanshu, let me know if you run into any issues with this new feature.

Hey @stephenberry added few thoughts at: #687, thanks!

from glaze.