flightaware / yajl-tcl Goto Github PK

There are missing docs, which causes the install to fail (but after the important files have already been installed):

# sudo make -k install
 /usr/bin/install -c libyajltcl.so.1 /usr/local/lib/yajltcl1.2/libyajltcl.so.1
 : /usr/local/lib/yajltcl1.2/libyajltcl.so.1
 Install yajl.tcl /usr/local/lib/yajltcl1.2/yajl.tcl
 Install pkgIndex.tcl /usr/local/lib/yajltcl1.2
Installing header files in /usr/local/include
Installing documentation in /usr/local/share/man
Installing ./doc/*.n
install: ./doc/*.n: No such file or directory
*** Error code 71 (continuing)
`install' not remade because of errors.
1 error

Just wanted to let you know that the language grammar that I have been working on for Atom Editor (and technically Sublime) will support highlighting for yajl-tcl! Love this package, thanks a ton! Wish we had a combination of this and rl_json in one (rl_json is a bit faster but it is not good at dynamic values / building up values).

For best results, it does require to use the undocumented map_key rather than "string" when mapping each key although it just highlights as as a string if map_key isn't used so not a big deal.

It is available in atom editor as "language-dashtcl" although these updates aren't uploaded yet as of writing this.

Anyway, definitely makes it a bit nicer to read and work with so thought I would share!

When trying to round-trip the result of parse back into a json object, i get:

bad option "boolean": must be array_open, array_close, bool, clear, double, integer, map_close, map_open, null, number, string, map_key, free, get, reset, delete, parse, or parse_complete

The yajl 1.x branch is now considered "legacy", as per the notation on https://github.com/lloyd/yajl/tree/1.x

The current development is https://github.com/lloyd/yajl and seems to change some of the API names around.

The "json" package http://tcllib.sourceforge.net/doc/json.html offers a ::json::json2dict method that outputs an easy to parse key/value list that can be loaded directly into a dict or an array to access individual elements:

{Country US Latitude 37.7668 precision zip State CA City {SAN FRANCISCO} Address {} Zip 94107 Longitude -122.3959} {Country US Latitude 37.371991 precision zip State CA City SUNNYVALE Address {} Zip 94085 Longitude -122.026020}

However, yajltcl's parse currently returns this, which is ponderously complex to mechanically navigate and extract just the value you want:

array_open map_open map_key precision string zip map_key Latitude number 37.7668 map_key Longitude number -122.3959 map_key Address string {} map_key City string {SAN FRANCISCO} map_key State string CA map_key Zip string 94107 map_key Country string US map_close map_open map_key precision string zip map_key Latitude number 37.371991 map_key Longitude number -122.026020 map_key Address string {} map_key City string SUNNYVALE map_key State string CA map_key Zip string 94085 map_key Country string US map_close array_close

Hey There,
Thanks so much for this package, I use it every day of my life pretty much! One thing that might be nice to add to this package is an official means of doing "raw" or "map_json" when already formatted jsonshould be added to what you are working on.

I am using number to achieve it now, and it's no big deal but would be nice to either add a note in the documentation or have some sort of official means of handling this scenario.

 proc formatRequest {identity method path messageBody} {
     set json [yajl create #auto]
     $json map_open \
         string "identity" string $identity                              \
         string "requests" array_open                                    \
        map_open                                                    \
            string "uuid" string $::URC::Controller                 \
            string "method" string $method                          \
            string "path" string $path                              \
            string "dataFormat" string "json"                       \
            string "messageBody" number $messageBody                \
            map_close                                                   \
         array_close                                                     \
     map_close                                                           
  set body [$json get]
  $json delete
  return $body


 }

Resulting in:

 {"identity":"1","requests":[{"uuid":"4","method":"2","path":"3","dataFormat":"json","messageBody":{"systemID":"4","key":"myKey"}}]}

Although json2dict is mentioned in the quick reference section of the README.md file in the repo, it isn't mentioned in the documentation page at https://flightaware.github.io/yajl-tcl/

There might be some other things that should be synced between the two as well.

The json2dict treats the unquoted string values true and false as boolean and converts them to 1 and 0.
Example:
{
"valueTrue": true,
"valueFalse": false
}

will be converted to:

valueTrue 1 valueFalse 0

Is there any configuration to prevent that ?

Thanks in advance!

Fix:
Including /usr/lib/x86_64-linux-gnu/libyajl.so (wherever this library happens to be installed...edit: dynamically discover this - thanks @resuna!) in Makefile.in
gcc -shared -pipe -O2 -fomit-frame-pointer -Wall -fPIC -Wl,--export-dynamic -o libyajltcl1.6.2.so yajltcl.o tclyajltcl.o yajltcllex.o /usr/lib/x86_64-linux-gnu/libyajl.so -L/usr/lib/x86_64-linux-gnu -ltclstub8.6
or running the above manually before running "sudo make install"

edit: On loading via "package require yajltcl" I would get: yajl_gen_array_close tcl undefined error.

/home/brad/yajl-tcl# ./configure
-su: ./configure: No such file or directory

In the fix for issue #4 it was pointed out that it was still possible for the caller to specify invalid, non-numeric arguments to the "number" command, which would allow invalid JSON to be generated.

The official specification in RFC7159 defines the grammar for the allowed numeric types:
https://tools.ietf.org/html/rfc7159#page-7

Hi,

We encoutered an error while trying to convert a json containing braces:

::yajl::json2dict { {"Comment": "ac\{db"} }

Result:

lexical error: inside a string, '\' occurs before a character which it may not. {"Comment": "ac\{db"} (right here) ------^

Also escaping the baces like: ::yajl::json2dict { {"Comment": "ac\\{db"} } does not help..

Any ideas or hints on that ?

Thanks in Advance

KR,
Rami

Ok so moving my code base over to not using the [number] work around i discuss in #21 and i am getting errors that make zero sense

set json [yajl create #auto]
set parsed [$json parse {{"persist":1}}]

parse error: trailing garbage
                                        {"persist":1}
                     (right here) ------^

From examining the yajl source and docs, if a non-null number callback function is defined in a yajl_callbacks structure and subsequently passed to yajltcl_make_parser, when the parser is used, the integer and double callbacks will never be invoked.

We should be able to change the number_callback in the "yajl_callbacks callbacks" structure definition to NULL and delete the number_callback function and gain a slight performance improvement in the "parse" method.

It's an easy fix but it could stand to be tested.

The parse2json method defines a number callback but does not define integer or double callbacks, so no change is needed there.

Hello! I needed to parse a collection of large JSON files and performance of the pure-Tcl json::json2dict was unsatisfactory.

Unaware of yajl-tcl I wrote my own -- which uses json-c for the actual heavy-lifting the way you are using yajl.

Only after I was done did it occur to me to search for existing C-implementations of JSON-parsing -- and I found yours.

I then compared the performance and now have the following numbers. All tests used the following script on 12 JSON-files (total of over 60Mb):

foreach f $argv {
    set fd  [open $f]
    set d   [yajl::json2dict [read $fd]]
    close $fd
    if {![dict exists $d users]} continue
    foreach user [dict get $d users] {
        array set a $user
        if {[info exists users($a(name))]} continue
        set users($a(name)) $a(fullname)
    }
}

This is, actually, what I needed to do -- extract the "users" part of all JSON-files, collect all such users into an array and print the array at the end.

The performance, as reported by tcsh's time-command (note the times and the memory-use):

As you can see, the json-c based implementation is dramatically faster than both the Pure TCL and the yajl based ones, even if it uses some more memory than the latter. I doubt, there is anything magic about my code -- the performance differences are, likely, attributable to the differences in the underlying JSON-parsers (json-c vs. yajl).

Maybe, json-c is using a hash-table, where yajl (or you?) are using a regular array? This would explain the higher memory use...

In any case, this is something you may wish to investigate closer.

Edit: tests where done with tclsh8.6 as provided by FreeBSD lang/tcl86 port on a FreeBSD-9.2/i386.

As stated as the only solution to #21 (which i still think needs a better solution/option for edge cases), the parse method would be the only way to handle this.

Based on what [$json parse] does, I would thin kit makes sense to expose the command without first creating an object. It doesn't seem to affect the actual object itself and takes a normal json value so in that sense I would propose

set parsed [yajl parse {{"one": "two"}}]

as an option to parse the json even when you have no need at that time for an actual yajl object.

If the "number" type is used to generate JSON, then yajl-tcl does not detect and reject a blank argument, making it possible to emit invalid JSON code.

#!/usr/local/bin/tclsh8.5
package require yajltcl
set x [yajl create #auto]
$x array_open
$x string type number 12 number 34 number "" number 56
$x array_close
puts [$x get]