Giter Club home page Giter Club logo

Comments (10)

stephenberry avatar stephenberry commented on August 26, 2024

This is strange behavior and it doesn't conform to "the shortest possible representation", which is what the underlying algorithm is supposed to do. Glaze uses a slightly modified version of the code yyjson uses, and looking over the code I would expect this to also show up in yyjson, which is interesting.

Will have to be investigated further. But, the output is valid JSON and correct, so it isn't a major concern.

from glaze.

pauldreik avatar pauldreik commented on August 26, 2024

I do not think it is valid json. Looking at the railroad diagram on https://www.json.org/json-en.html it seems like a leading 0 must be followed by an optional fraction and optional exponent.

Also, it is rejected when deserializing back from json - I discovered this as part of a roundtrip test. So I think this should be labeled bug, not enhancement.

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

Yup, my bad. You're right this is a bug and broken.

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

Note: This case works for double, just not float. The original code from yyjson only handles double, so the optimizations for float using double parsing code introduce some incompatibility. Looking into it.

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

I'm moving to use std::to_chars with special conversions for inf and nan to null for JSON compliance. This should fix any serialization issues with floating point numbers.

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

Well... the performance for std::to_chars is abysmal at least on MacOS Arm. I'm going to figure out a faster solution, it just might take a bit longer.

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

I've switched Glaze to use the dragonbox algorithm underneath with new float handling. It solves this particular case and I haven't gotten it to error in my tests.

from glaze.

pauldreik avatar pauldreik commented on August 26, 2024

Unfortunately this still does not handle my roundtrip tests. It failed on the denormalized number 1.40129846e-45f which got serialized to {"value":1.E-45} which is rejected by the glaze deserializer.

from glaze.

pauldreik avatar pauldreik commented on August 26, 2024

It is work in progress, but you can view that particular test here: https://github.com/pauldreik/glaze/blob/fuzz/fuzzing/json_roundtrip_floating.cpp

from glaze.

stephenberry avatar stephenberry commented on August 26, 2024

Thanks for sharing your fuzz testing! Also, I've added better random float and double testing to Glaze in #1190.

#1190 also fixes the denormalized number serialization issue.

from glaze.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.