vivkin / gason Goto Github PK

Your json library did well in this benchmark I happened to come across. Its the fastest C++11 json library. (I'm using the cpprest json library at the moment, and am thinking of switching.)

However the "conformance" on your library was below average. Maybe you can do something to improve it.

https://github.com/miloyip/nativejson-benchmark

dear vivkin;

I enjoyed your parser a lot esp for its low memory footprint.

To mold the api into more c++ (i don't mean std::) and give the end-user even more simple API I've made some small changes in gason to form gason++

• using gason namespace
• add a few method for finding children and array items.
• overloading operators for validity and type (tag) checking and ...
• adding a simple JSon builder class.

because these changes breaks your API (just using namespace gason; should be added to legacy codes) I prefer not to fork and submit an immediate pull request.

but if you find these changes useful please merge them into gason:
https://github.com/azadkuh/gason++

it's better to add:

allocator.deallocate();

to the bigining of jsonParse() to stop allocator instance growing every time if the instance has been used before.

users may forget to call deallocate() or introduce a new instance of allocator every time they call jsonParse().

Android is starting to use tagged pointers (https://source.android.com/devices/tech/debug/tagged-pointers)

This code breaks with tagged pointers:

A real life example that occurred on a Pixel 5 is of a JsonValue created with the tag 2, and with the payload 0xb400007a779ebdc0.
Calling getTag() on that value returns 0, but it should have returned 2.

In this line https://github.com/vivkin/gason/blob/master/src/gason.h#L33
on 32 bit systems If payload's highest bit is set eg ( 0x8015b678 ) then (uint64_t)payload == 0xffffffff8015b678 so the payload bits overwrites the tags and nan_mask.

masking the result of ((uint64_t)payload) solves the issue.
ival = JSON_VALUE_NAN_MASK | ((uint64_t)tag << JSON_VALUE_TAG_SHIFT) | (((uint64_t)payload)&JSON_VALUE_PAYLOAD_MASK)

I tested this on win32 and android (32bit).

JsonValue save the address of pointer, but if the value is std::string, and maybe it is a temporary variables, in this situation, the address is held by JsonValue is valid.

how to reslove the problem?

In JavaScript (NodeJS) Number.MIN_VALUE is 5e-324 and MAX_VALUE 1.7976931348623157e+308.

These numbers are parsed by gason as 0 and infinity respectively. C++ will handle these as literals quite happily so string2double must be causing an underflow and overflow.

I replicated this case with simple mods to test-suite.cpp on master:

void parse(const char *csource, bool ok) {
    char *source = strdup(csource);
    char *endptr;
    JsonValue value;
    JsonAllocator allocator;
    int result = jsonParse(source, &endptr, &value, allocator);
    printf("result is %d, value is: %g\n", result, value.fval);
    ++parsed;
    free(source);
}

int main() {
    pass(u8R"json(5e-324)json");
    pass(u8R"json(1.7976931348623157e+308)json");
    return 0;
}

The output is:

result is: 0, value is: 0
result is: 0, value is: inf

Working case example:

int main()
{
  double min = 5e-324;
  double max = 1.7976931348623157e+308;
  printf("Min: %g, Max: %g\n", min, max);
}

This program prints 'Min: 4.94066e-324, Max: 1.79769e+308'.

I see this problem on an Intel i7 under Debian 8. I guess on some other CPU architectures this test may not fail in this manner.

Hi,

First off - many thanks for such a great library.

when using for (auto i : obj) to loop through an array (JSON_ARRAY), are we guaranteed for the order? Docs don't seem to specify it so wanted to check.

Many thanks,
Ege

Greetings,

I'm trying this gasoncpp on an Odroid-C1 and get the following error while parsing a json file:

root@heidenrod-obs:/Astro/obs-dev/gason# ./gasonpp canada.json
gasonpp: /Astro/obs-dev/gason/src/gason.h:32: JsonValue::JsonValue(JsonTag, void*): Assertion `(uint64_t)payload <= 0x00007FFFFFFFFFFFULL' failed.
root@heidenrod-obs:/Astro/obs-dev/gason#

Here's the full gdb bt:

root@heidenrod-obs:/Astro/obs-dev/gason# gdb --eval-command="set follow-fork-mode child" --args ./gasonpp canada.json
GNU gdb (Ubuntu 7.9-1ubuntu1) 7.9
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./gasonpp...done.
(gdb) run
Starting program: /Astro/obs-dev/gason/gasonpp canada.json
gasonpp: /Astro/obs-dev/gason/src/gason.h:32: JsonValue::JsonValue(JsonTag, void*): Assertion `(uint64_t)payload <= 0x00007FFFFFFFFFFFULL' failed.

Program received signal SIGABRT, Aborted.
__libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
47 ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.
(gdb) bt
#0 __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1 0xb6f0062e in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#2 0xb6f01332 in __GI_abort () at abort.c:89
#3 0xb6efb98a in __assert_fail_base (fmt=0x1 <error: Cannot access memory at address 0x1>,

assertion=0x12ea4 "(uint64_t)payload <= 0x00007", 'F' <repeats 11 times>, "ULL", 
assertion@entry=0x2 <error: Cannot access memory at address 0x2>, file=file@entry=0xb6ffa390 "", line=32, line@entry=3069997192, 
function=function@entry=0x13084 <JsonValue::JsonValue(JsonTag, void*)::__PRETTY_FUNCTION__> "JsonValue::JsonValue(JsonTag, void*)")
at assert.c:92

#4 0xb6efba22 in __GI___assert_fail (assertion=0x2 <error: Cannot access memory at address 0x2>, file=0xb6ffa390 "", line=3069997192,

function=0x13084 <JsonValue::JsonValue(JsonTag, void*)::__PRETTY_FUNCTION__> "JsonValue::JsonValue(JsonTag, void*)") at assert.c:101

#5 0x00011994 in JsonValue::JsonValue (this=0xbefff498, tag=JSON_STRING, payload=0xb671700b) at /Astro/obs-dev/gason/src/gason.h:32
#6 0x00012474 in jsonParse (

s=0xb671700b "type\": \"FeatureCollection\",\r\n  \"features\": [\r\n{\r\n    \"type\": \"Feature\",\r\n\"properties\": { \"name\": \"Canada\" },\r\n\"geometry\": {\"type\":\"Polygon\",\"coordinates\":[[[-65.613616999999977,43.420273000000009],[-6"..., endptr=0xbefff64c, value=0xbefff668, 
allocator=...) at /Astro/obs-dev/gason/src/gason.cpp:175

#7 0x000118d4 in main (argc=2, argv=0xbefff7e4) at /Astro/obs-dev/gason/src/pretty-print.cpp:183

(gdb)

Info about this little board:
root@heidenrod-obs:/Astro/obs-dev/gason# uname -a
Linux heidenrod-obs 3.10.66-49 #1 SMP PREEMPT Fri Jan 30 22:17:03 BRST 2015 armv7l armv7l armv7l GNU/Linux
root@heidenrod-obs:/Astro/obs-dev/gason# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 15.04
Release: 15.04
Codename: vivid
root@heidenrod-obs:/Astro/obs-dev/gason#

I was wondering if you could help me out there please?

I realise that // and /* ... */ comments are both not part of JSON, but they commonly pop up. I feel it it fit's gason's aim to parse as best as possible to handle comments, and they would not hurt efficiency.

I am happy to knock up a patch for this, but wanted to check before I bothered if it would in principle be accepted. I would make no attempt at saving the comments or making them accessable, just skip straight on over them.

Is there supposed to be a fallthrough on this line?

        *endptr = s++;
        switch (**endptr) {
        case '-':
            if (!isdigit(*s) && *s != '.') {
                *endptr = s;
                return JSON_BAD_NUMBER;
            }
        // Should I fallthrough here?
        case '0':
        case '1':
        case '2':
        case '3':

https://github.com/miloyip/nativejson-benchmark/blob/master/sample/conformance_gason%20(C%2B%2B11).md

Hello,

while adding testcases and removing duplicated found, you have some switched - pass/fail also when checking by online validator (https://jsonformatter.curiousconcept.com/).

Is there any reason for that (except some valid RFC are a bit crazy, etc.) ?

In case - my C# translation has almost same logic, but switched test cases are fixed to end with proper results.

Fork here:
https://github.com/eltomjan/Gason

This code does not compile in MS Visual Studio 2012 C++:
inline JsonIterator begin(JsonValue o) {
return JsonIterator{o.toNode()};
}

(Compiled OK in MS Viaual Studio 2015 Community + Update 1,
but I use VS 2012 C++ for my project as VS 2015 Community is buggy.)

Can you please fix it or can you describe how to fix it by myself to compile it on VS 2012?

Best regards,
Alexander

P.S. And please replace one known 3-letter Russian word from test-suite.cpp, bad style and "plokhaya primeta, programma budet plokho rabotat'" .

Hi.

I have been playing around with afl fuzzing and the attached input file seems to provoke a heap buffer overflow in gason.
I have not investigated in depth, but afl-analyze suggests that handling of backslash in input is to blame.

gason.crash.00.min.json.txt

• char8_t is C++20 keyword
• json mandates utf-8
• gason needs to work with char8_t *
• is that even possible?

Right now the jsonParse() actually mutates the input buffer when handling backslash escaping.

Would be nice if it didn't do that, and it typically counters what the user expects when using a parser.

Hello,

there is no unicode escape processing ?

Btw, made a C# translation - a bit slower, bit more memory used, but raw POC seems to work:
https://github.com/eltomjan/ETEhomeTools/tree/master/CSharp/GasonC%23

Anyway gason is very nice and quick parser...

P.S. C# translates them automatically in string (ex. \u0037 -> 7), but you can force them (\uXXXX) or read from file in case(?).

Current implementation has inaccurate number parsing.

How to Read Floating Point Numbers Accurately by William D. Clinger - a classic paper on number parsing.

This issue is an extension of #19

The function string2double introduce a precision error when an exponent expression is parsed.
The floating result should be computed only once at the end from an integer rather than adjusting the floating result in each iteration.

It should be something like that:

long long number = 0;
bool is_negative = false;
int scale = 0;

// parse sign
if (*s == '-' || *s == '+'){
  is_negative = *s == '-';
  ++s;
}

// parse integer part
while (isdigit(*s))
    number = (number * 10) + (*s++ - '0');

// parse fraction part
if (*s == '.') {
    ++s;
    while (isdigit(*s)) {
        --scale;
        number = (number * 10) + (*s++ - '0');
    }
}

// set sign
if (is_negative)
  number = -number;

// parse exponent part
if (*s == 'e' || *s == 'E') {
  ++s;
  
  bool is_exponent_negative = false;
  if (*s == '+' || *s == '-'){
    is_exponent_negative = *s == '-';
    ++s;
  }
  
  int exponent = 0;
  while (isdigit(*s))
    exponent = (exponent * 10) + (*s++ - '0');

  if (is_exponent_negative)    
    exponent = -exponent;
  
  scale += exponent;
}

// adjust scale
double result = (double)number;
if (scale != 0)
  result *= pow(10, scale);

Hi,

I have been using the gason tool to flash set of binaries i.e., .FLS files.
In this process using older vjson binary worked absolutely fine, but the latest Gason binaries gave the error as :
Thread 1 "downloadtool" received signal SIGSEGV, Segmentation fault.
__strlen_avx2 () at ../sysdeps/x86_64/multiarch/strlen-avx2.S:65
65 ../sysdeps/x86_64/multiarch/strlen-avx2.S: No such file or directory.

Thanks in advance for any help on why this error.

When parsing numbers with big negative exponent, such as [1e-21474836311], gason gets into infinity loop.