python-rapidjson / python-rapidjson Goto Github PK

View Code? Open in Web Editor NEW

495.0 10.0 49.0 1.83 MB

Python wrapper around rapidjson

License: Other

C++ 60.24% Python 38.72% Emacs Lisp 0.08% Makefile 0.96%

python3 json-serialization json-parser

python-rapidjson's Introduction

python-rapidjson

Python wrapper around RapidJSON

Authors:	Ken Robbins <[email protected]> Lele Gaifax <[email protected]>
License:	MIT License
Status:

RapidJSON is an extremely fast C++ JSON parser and serialization library: this module wraps it into a Python 3 extension, exposing its serialization/deserialization (to/from either bytes, str or file-like instances) and JSON Schema validation capabilities.

Latest version documentation is automatically rendered by Read the Docs.

Getting Started

First install python-rapidjson:

$ pip install python-rapidjson

or, if you prefer Conda:

$ conda install -c conda-forge python-rapidjson

Basic usage looks like this:

>>> import rapidjson
>>> data = {'foo': 100, 'bar': 'baz'}
>>> rapidjson.dumps(data)
'{"foo":100,"bar":"baz"}'
>>> rapidjson.loads('{"bar":"baz","foo":100}')
{'bar': 'baz', 'foo': 100}
>>>
>>> class Stream:
...   def write(self, data):
...      print("Chunk:", data)
...
>>> rapidjson.dump(data, Stream(), chunk_size=5)
Chunk: b'{"foo'
Chunk: b'":100'
Chunk: b',"bar'
Chunk: b'":"ba'
Chunk: b'z"}'

Development

If you want to install the development version (maybe to contribute fixes or enhancements) you may clone the repository:

$ git clone --recursive https://github.com/python-rapidjson/python-rapidjson.git

Note

The --recursive option is needed because we use a submodule to include RapidJSON sources. Alternatively you can do a plain clone immediately followed by a git submodule update --init.

Alternatively, if you already have (a compatible version of) RapidJSON includes around, you can compile the module specifying their location with the option --rj-include-dir, for example:

$ python3 setup.py build --rj-include-dir=/usr/include/rapidjson

A set of makefiles implement most common operations, such as build, check and release; see make help output for a list of available targets.

Performance

python-rapidjson tries to be as performant as possible while staying compatible with the json module.

See the this section in the documentation for a comparison with other JSON libraries.

Incompatibility

Although we tried to implement an API similar to the standard library json, being a strict drop-in replacement in not our goal and we have decided to depart from there in some aspects. See this section in the documentation for further details.

python-rapidjson's People

Contributors

Stargazers

Watchers

python-rapidjson's Issues

Failed to install this package from source code on Win10

Hey, guys!
I encountered several compilation errors while installing from source on my PC.

My environment:
OS: Win10 Simplified Chinese(So the default system encoding is GBK, not UTF-8)
Python version: Python 3.6
VS2015(vc14) is also installed on my PC.

The first error is about setup.py.

D:\GitHub\python-rapidjson>py -3 setup.py install
Traceback (most recent call last):
File "setup.py", line 27, in
long_description = f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa6 in position 4092: illegal multibyte sequence

After I changed code below:

with open('README.rst') as f:
    long_description = f.read()

with open('README.rst', encoding='utf-8') as f:
    long_description = f.read()

This error disappeared.

The second error is about python-rapidjson/rapidjson.cpp:

...
running build_ext
building 'rapidjson' extension
D:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I./rapidjson/include "-ID:\Program Files\Python36\include" "-ID:\Program Files\Python36\include" "-ID:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-ID:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tp./python-rapidjson/rapidjson.cpp /Fobuild\temp.win-amd64-3.6\Release./python-rapidjson/rapidjson.obj
rapidjson.cpp
./python-rapidjson/rapidjson.cpp(341): error C2131: expression did not evaluate to a constant
./python-rapidjson/rapidjson.cpp(341): note: failure was caused by non-constant arguments or reference to a non-constant symbol
./python-rapidjson/rapidjson.cpp(341): note: see usage of 'length'
./python-rapidjson/rapidjson.cpp(344): error C3863: array type 'char [length+]' is not assignable

So I had to change code:

            char zstr[length + 1];

            strncpy(zstr, str, length);
            zstr[length] = '\0';

            value = PyLong_FromString(zstr, NULL, 10);

            std::string zstr(str, length);

            value = PyLong_FromString(zstr.c_str(), NULL, 10);

The last error I encountered:

./python-rapidjson/rapidjson.cpp(1060): error C2556: 'char (*__timezone(void))[10]': overloaded function differs only by return type from 'long *__timezone(void)'
C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\time.h(73): note: see declaration of '__timezone'
./python-rapidjson/rapidjson.cpp(1060): error C2373: '__timezone': redefinition; different type modifiers
C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\time.h(73): note: see declaration of '__timezone'

It seems that timezone is defined as a macro. So, again MSVC failed to compile.
So, I renamed variable timezone to timeZone.

bundled wheel?

Hi,

Any chance on binary wheel package which doesn't require compilation? I have no problems compiling it on my box, but some machines are a bit strict regarding installing compilers and stuff

--prompt for venv ?

It would be great to have some info for what kind of system you require. In my tests, the Makefiles fail because when trying to set up python/venv they use --promot which is not available as a commandline option. When I checked the docs, it seems like this is an API function but not necessarily available as a commandline argument. This is on Ubuntu 16.04.3 with an up to date python installation.

Add support for `ensure_ascii` kwarg

This is True by default so I think its important

Add support for `sort_keys` kwarg

Add support for `use_decimal` kwarg

experiment with releasing the GIL on python-rapidjson so it could be faster across threads

0.7 release package seems to be missing the rapidjson C sources

Pip install bails our as it's missing a rapidjson header

Collecting python-rapidjson
  Using cached python-rapidjson-0.0.7.tar.gz
Building wheels for collected packages: python-rapidjson
  Running setup.py bdist_wheel for python-rapidjson ... error
  Complete output from command /usr/local/opt/python3/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/k8/92852d8j4qb73d8ch8dzd61c0000gn/T/pip-build-u0fjfury/python-rapidjson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/k8/92852d8j4qb73d8ch8dzd61c0000gn/T/tmpoeh3nkjwpip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_ext
  /usr/local/lib/python3.5/site-packages/Cython/Distutils/old_build_ext.py:30: UserWarning: Cython.Distutils.old_build_ext does not properly handle dependencies and is deprecated.
    "Cython.Distutils.old_build_ext does not properly handle dependencies "
  building 'rapidjson' extension
  creating build
  creating build/temp.macosx-10.11-x86_64-3.5
  creating build/temp.macosx-10.11-x86_64-3.5/python-rapidjson
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I./rapidjson/include -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c ./python-rapidjson/rapidjson.cpp -o build/temp.macosx-10.11-x86_64-3.5/./python-rapidjson/rapidjson.o
  ./python-rapidjson/rapidjson.cpp:9:10: fatal error: 'rapidjson/reader.h' file not found
  #include "rapidjson/reader.h"
           ^
  1 error generated.
  error: command 'clang' failed with exit status 1```

accept and ignore `separators` kwarg

Currently if a separators kwarg is passed to dumps a TypeError is thrown ("'separators' is an invalid keyword argument for this function").

Callers don't always have the ability to suppress separators from being passed (e.g. pallets/flask#1602).

Instead of throwing when separators is passed, would you consider just ignoring it? If it were used for something more important, you could make the argument that it's better to fail loudly than silently, but given the above, and that separators is only used for pretty printing, I think just documenting that separators is ignored is reasonable. What do you think?

Install latest version on Mac OS

Getting gcc error when I try to install latest version on Mac OS:

./rapidjson.cpp:1653:17: error: use of undeclared identifier 'isnan'; did you mean 'std::isnan'?
              if (Py_IS_NAN(d)) {

./rapidjson.cpp:1661:24: error: use of undeclared identifier 'isinf'
              } else if (Py_IS_INFINITY(d)) {

....


 ./rapidjson.cpp:2246:61: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]
         T_UINT, offsetof(EncoderObject, numberMode), READONLY, "number_mode"},
                                                                ^
24 warnings and 2 errors generated.
error: command 'gcc' failed with exit status 1

cannot install on windows 7

C:\Users\florinb>python -3.4 -m pip install python-rapidjson
Unknown option: -.
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.

C:\Users\florinb>python -3 -m pip install python-rapidjson
C:\Python27\lib\site-packages\pip_vendor\requests\auth.py:44: DeprecationWarning: Overriding eq blocks inheritance of _hash
class HTTPBasicAuth(AuthBase):
C:\Python27\lib\site-packages\pip_vendor\requests\auth.py:73: DeprecationWarning: Overriding eq blocks inheritance of _hash
class HTTPDigestAuth(AuthBase):
C:\Python27\lib\site-packages\ipaddress.py:82: DeprecationWarning: Overriding eq blocks inheritance of hash in 3.x
class _TotalOrderingMixin(object):
Collecting python-rapidjson
Using cached python-rapidjson-0.2.3.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "c:\users\florinb\appdata\local\temp\pip-build-13fr8v\python-rapidjson\setup.py", line 28, in
raise NotImplementedError("Only Python 3+ is supported.")
NotImplementedError: Only Python 3+ is supported.

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in c:\users\florinb\appdata\local\temp\pip-build-13fr8v\python-rapidjs

Support ``indent`` kwargs with the pretty print writer

Add support for kwarg `allow_nan`

Pip installation fails

Since release 0.0.9 pip installation fails on python 3.5.1:

Collecting python-rapidjson
  Downloading python-rapidjson-0.0.9.tar.gz (126kB)
    100% |████████████████████████████████| 133kB 845kB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/x6/dp4jcb4j0v10f_6tvgr4dr200000gn/T/pip-build-3yn_8369/python-rapidjson/setup.py", line 43, in <module>
        with open('CHANGES.rst', encoding='utf-8') as f:
    FileNotFoundError: [Errno 2] No such file or directory: 'CHANGES.rst'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/x6/dp4jcb4j0v10f_6tvgr4dr200000gn/T/pip-build-3yn_8369/python-rapidjson/

Add native lib benchmarks docs

this is a nice benchmark to show why rapid json is good

https://github.com/miloyip/nativejson-benchmark/blob/master/README.md

Support for Python 2.7

I saw the statement that this project doesn't support "legacy versions of Python" in the README. However, many companies still rely on Python 2.7 in production systems and having tooling that supports both 2 and 3 provides a better upgrade path.

Is it trivial for you to also support 2.7? Would you please consider adding that support so forks/patches are not necessary? If you're not willing to do the work yourself, would you accept a PR that adds this support?

document loads and dumps

document loads and dumps and their parameters

Null-bytes are stripped out

Hello, I found that '\x00' is encoded as ''
It represents an incompatibily with the standard json module, which maintains it as '\x00'.
Thanks

slow encoding

We should look at why this slowed down so much. I think the decoding is still fine even though it got slower, the encoding is even slower than simplejson.

See #30

Install and Build fails for OSX

Building or installing python-rapidjson on OSX currently fails with the following error:

./rapidjson.cpp:560:25: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
            std::string zstr(str, length);

If one edits the rapidjson.cpp file and adds the following additional imports:

#include <string>
#include <sstream>
#include <iostream>

then the build will now succeed.

If acceptable, I can generate a PR with these changes added. If I do this, should I scope these additions to OSX/Darwin build environments?

segfault on surrogate unicode

Python 3.6.1 (default, Mar 23 2017, 02:34:11)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> import rapidjson
>>> rapidjson.__version__
'0.2.4'
>>> foo = {"foo": "\ud80d"}
>>> json.dumps(foo)
'{"foo": "\\ud80d"}'
>>> rapidjson.dumps(foo)
Segmentation fault (core dumped)

cant install on OSX Maverick (10.9.5) python 3.4.3

After pip install python-rapidjson
The following output happens
Collecting python-rapidjson
Using cached python-rapidjson-0.0.11.tar.gz
Installing collected packages: python-rapidjson
Running setup.py install for python-rapidjson ... error
Complete output from command /Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4 -u -c "import setuptools, tokenize;file='/private/var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T/pip-build-4fiy0ttq/python-rapidjson/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T/pip-8yx2m6bb-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'rapidjson' extension
creating build
creating build/temp.macosx-10.6-intel-3.4
creating build/temp.macosx-10.6-intel-3.4/python-rapidjson
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch i386 -arch x86_64 -g -I./rapidjson/include -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c ./python-rapidjson/rapidjson.cpp -o build/temp.macosx-10.6-intel-3.4/./python-rapidjson/rapidjson.o -pedantic
In file included from /Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/Python.h:78,
from ./python-rapidjson/rapidjson.cpp:1:
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:88: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:89: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:90: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:91: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:92: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:93: error: ISO C++ does not support 'long long'
In file included from ./rapidjson/include/rapidjson/writer.h:21,
from ./python-rapidjson/rapidjson.cpp:11:
./rapidjson/include/rapidjson/internal/dtoa.h:32: warning: unknown option after '#pragma GCC diagnostic' kind
In file included from /Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/Python.h:78,
from ./python-rapidjson/rapidjson.cpp:1:
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:88: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:89: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:90: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:91: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:92: error: ISO C++ does not support 'long long'
/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/longobject.h:93: error: ISO C++ does not support 'long long'
In file included from ./rapidjson/include/rapidjson/writer.h:21,
from ./python-rapidjson/rapidjson.cpp:11:
./rapidjson/include/rapidjson/internal/dtoa.h:32: warning: unknown option after '#pragma GCC diagnostic' kind
fatal error: /Library/Developer/CommandLineTools/usr/bin/lipo: can't figure out the architecture type of: /var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T//cccQX3Nv.out
error: command 'gcc-4.2' failed with exit status 1

----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.4/bin/python3.4 -u -c "import setuptools, tokenize;file='/private/var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T/pip-build-4fiy0ttq/python-rapidjson/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T/pip-8yx2m6bb-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/cm/dwsx5_yn5p92nzfyrn8g68wr0000gn/T/pip-build-4fiy0ttq/python-rapidjson/

Have tried to uninstall XCode and install Command Line tools, didnt help.
Currently

gcc --version

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

0.0.9 fails to install - FileNotFoundError: [Errno 2] No such file or directory: 'CHANGES.rst'

$ pip install python-rapidjson
Collecting python-rapidjson
  Using cached python-rapidjson-0.0.9.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-q9nkddjq/python-rapidjson/setup.py", line 43, in <module>
        with open('CHANGES.rst', encoding='utf-8') as f:
    FileNotFoundError: [Errno 2] No such file or directory: 'CHANGES.rst'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-q9nkddjq/python-rapidjson/

Add support for `separators` kwarg

Intallation error on latest git version

Windows:

Collecting https://github.com/kenrobbins/python-rapidjson/archive/master.zip
  Downloading https://github.com/kenrobbins/python-rapidjson/archive/master.zip
     / 40kB 204kB/s
Installing collected packages: python-rapidjson
  Running setup.py install for python-rapidjson ... error
    Complete output from command d:\program_files\python_3.6\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\martijn\\AppData\\Local\\Temp\\pip-hkd_uxv6-build\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\martijn\AppData\Local\Temp\pip-q5huswg_-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'rapidjson' extension
    creating build
    creating build\temp.win32-3.6
    creating build\temp.win32-3.6\Release
    creating build\temp.win32-3.6\Release\python-rapidjson
    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I./rapidjson/include -Id:\program_files\python_3.6\include -Id:\program_files\python_3.6\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" /EHsc /Tp./python-rapidjson/rapidjson.cpp /Fobuild\temp.win32-3.6\Release\./python-rapidjson/rapidjson.obj
    rapidjson.cpp
    ./python-rapidjson/rapidjson.cpp(9): fatal error C1083: Cannot open include file: 'rapidjson/reader.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\cl.exe' failed with exit status 2

    ----------------------------------------
Command "d:\program_files\python_3.6\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\martijn\\AppData\\Local\\Temp\\pip-hkd_uxv6-build\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\martijn\AppData\Local\Temp\pip-q5huswg_-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\martijn\AppData\Local\Temp\pip-hkd_uxv6-build\

Linux subsystem (Win10):

Collecting https://github.com/kenrobbins/python-rapidjson/archive/master.zip
  Downloading https://github.com/kenrobbins/python-rapidjson/archive/master.zip
Installing collected packages: python-rapidjson
  Running setup.py install for python-rapidjson ... error
    Complete output from command /usr/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-nmn11f9r-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-u164wex7-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'rapidjson' extension
    creating build
    creating build/temp.linux-x86_64-3.5
    creating build/temp.linux-x86_64-3.5/python-rapidjson
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I./rapidjson/include -I/usr/include/python3.5m -c ./python-rapidjson/rapidjson.cpp -o build/temp.linux-x86_64-3.5/./python-rapidjson/rapidjson.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
    ./python-rapidjson/rapidjson.cpp:9:30: fatal error: rapidjson/reader.h: No such file or directory
     #include "rapidjson/reader.h"
                                  ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-nmn11f9r-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-u164wex7-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-nmn11f9r-build/

Support for stream (equivalent to json.load/json.dump)

We would like to be able to parse streams/file objects to avoid having to read first the entire stream and then parse it.

Schema validation support??

Any plans for adding the schema validation support from RapidJson??

https://github.com/miloyip/rapidjson/blob/master/doc/schema.md

Can't dump bytes

See #9.

Serialize bytes as if it were a UTF-8 encoded string.

Warnings during compile for v0.2.7

I was updating the conda-forge feedstock to 0.2.7 and noticed the following warnings in the builds for linux and os x. Full logs here:
https://circleci.com/gh/conda-forge/python-rapidjson-feedstock/tree/pull%2F6
https://travis-ci.org/conda-forge/python-rapidjson-feedstock/builds/314361977

cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
 };
 ^
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:1145:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
 };
 ^
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
./rapidjson.cpp:2250:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]

./rapidjson.cpp:1136:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"datetime_mode",

     ^

./rapidjson.cpp:1137:63: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(DecoderObject, datetimeMode), READONLY, "datetime_mode"},

                                                              ^

./rapidjson.cpp:1138:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"uuid_mode",

     ^

./rapidjson.cpp:1139:59: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(DecoderObject, uuidMode), READONLY, "uuid_mode"},

                                                          ^

./rapidjson.cpp:1140:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"number_mode",

     ^

./rapidjson.cpp:1141:61: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(DecoderObject, numberMode), READONLY, "number_mode"},

                                                            ^

./rapidjson.cpp:1142:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"parse_mode",

     ^

./rapidjson.cpp:1143:60: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(DecoderObject, parseMode), READONLY, "parse_mode"},

                                                           ^

./rapidjson.cpp:2233:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"skip_invalid_keys",

     ^

./rapidjson.cpp:2234:66: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_BOOL, offsetof(EncoderObject, skipInvalidKeys), READONLY, "skip_invalid_keys"},

                                                                 ^

./rapidjson.cpp:2235:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"ensure_ascii",

     ^

./rapidjson.cpp:2236:62: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_BOOL, offsetof(EncoderObject, ensureAscii), READONLY, "ensure_ascii"},

                                                             ^

./rapidjson.cpp:2237:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"indent",

     ^

./rapidjson.cpp:2238:57: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(EncoderObject, indent), READONLY, "indent"},

                                                        ^

./rapidjson.cpp:2239:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"sort_keys",

     ^

./rapidjson.cpp:2240:62: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_BOOL, offsetof(EncoderObject, ensureAscii), READONLY, "sort_keys"},

                                                             ^

./rapidjson.cpp:2241:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"max_recursion_depth",

     ^

./rapidjson.cpp:2242:68: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(EncoderObject, maxRecursionDepth), READONLY, "max_recursion_depth"},

                                                                   ^

./rapidjson.cpp:2243:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"datetime_mode",

     ^

./rapidjson.cpp:2244:63: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(EncoderObject, datetimeMode), READONLY, "datetime_mode"},

                                                              ^

./rapidjson.cpp:2245:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"uuid_mode",

     ^

./rapidjson.cpp:2246:59: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(EncoderObject, uuidMode), READONLY, "uuid_mode"},

                                                          ^

./rapidjson.cpp:2247:6: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

    {"number_mode",

     ^

./rapidjson.cpp:2248:61: warning: conversion from string literal to 'char *' is deprecated [-Wc++11-compat-deprecated-writable-strings]

     T_UINT, offsetof(EncoderObject, numberMode), READONLY, "number_mode"},

                                                            ^

24 warnings generated.

Compilation errors when installing from PyPi and git

Collecting git+https://github.com/kenrobbins/python-rapidjson
  Cloning https://github.com/kenrobbins/python-rapidjson to /tmp/pip-54hC76-build
Installing collected packages: python-rapidjson
  Running setup.py install for python-rapidjson
    Complete output from command /home/omer/.virtualenvs/playground/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-54hC76-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-RoOWJP-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/omer/.virtualenvs/playground/include/site/python2.7/python-rapidjson:
    running install
    running build
    running build_ext
    building 'rapidjson' extension
    creating build
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/python-rapidjson
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./thirdparty/rapidjson/include -I/home/omer/.pyenv/versions/2.7.10/include/python2.7 -c ./python-rapidjson/rapidjson.cpp -o build/temp.linux-x86_64-2.7/./python-rapidjson/rapidjson.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::NaN()’:
    ./python-rapidjson/rapidjson.cpp:199:43: error: too few arguments to function ‘PyObject* PyFloat_FromString(PyObject*, char**)’
                 value = PyFloat_FromString(str);
                                               ^
    In file included from /home/omer/.pyenv/versions/2.7.10/include/python2.7/Python.h:89:0,
                     from ./python-rapidjson/rapidjson.cpp:1:
    /home/omer/.pyenv/versions/2.7.10/include/python2.7/floatobject.h:48:24: note: declared here
     PyAPI_FUNC(PyObject *) PyFloat_FromString(PyObject*, char** junk);
                            ^
    ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::Infinity(bool)’:
    ./python-rapidjson/rapidjson.cpp:229:43: error: too few arguments to function ‘PyObject* PyFloat_FromString(PyObject*, char**)’
                 value = PyFloat_FromString(str);
                                               ^
    In file included from /home/omer/.pyenv/versions/2.7.10/include/python2.7/Python.h:89:0,
                     from ./python-rapidjson/rapidjson.cpp:1:
    /home/omer/.pyenv/versions/2.7.10/include/python2.7/floatobject.h:48:24: note: declared here
     PyAPI_FUNC(PyObject *) PyFloat_FromString(PyObject*, char** junk);
                            ^
    ./python-rapidjson/rapidjson.cpp: In function ‘PyObject* rapidjson_loads(PyObject*, PyObject*, PyObject*)’:
    ./python-rapidjson/rapidjson.cpp:342:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
         };
         ^
    ./python-rapidjson/rapidjson.cpp:342:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:342:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:342:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:342:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:366:66: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
             jsonStr = PyUnicode_AsUTF8AndSize(jsonObject, &jsonStrLen);
                                                                      ^
    ./python-rapidjson/rapidjson.cpp:401:47: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                     emsg = PyUnicode_AsUTF8(evalue);
                                                   ^
    ./python-rapidjson/rapidjson.cpp: In function ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode)’:
    ./python-rapidjson/rapidjson.cpp:513:68: error: there are no arguments to ‘PyUnicode_AsUTF8AndSize’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8AndSize’ must be available [-fpermissive]
                 char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                                        ^
    ./python-rapidjson/rapidjson.cpp:513:68: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
    ./python-rapidjson/rapidjson.cpp:571:46: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                 char* s = PyUnicode_AsUTF8(object);
                                                  ^
    ./python-rapidjson/rapidjson.cpp:607:61: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                             char* key_str = PyUnicode_AsUTF8(key);
                                                                 ^
    ./python-rapidjson/rapidjson.cpp:622:61: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                             char* key_str = PyUnicode_AsUTF8(key);
                                                                 ^
    ./python-rapidjson/rapidjson.cpp:721:95: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                 PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                                   ^
    ./python-rapidjson/rapidjson.cpp: In function ‘PyObject* rapidjson_dumps(PyObject*, PyObject*, PyObject*)’:
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
         };
         ^
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp:781:5: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
    ./python-rapidjson/rapidjson.cpp: At global scope:
    ./python-rapidjson/rapidjson.cpp:852:8: error: ‘PyModuleDef’ does not name a type
     static PyModuleDef rapidjson_module = {
            ^
    In file included from /usr/include/locale.h:28:0,
                     from /usr/include/c++/5/clocale:42,
                     from /usr/include/x86_64-linux-gnu/c++/5/bits/c++locale.h:41,
                     from /usr/include/c++/5/bits/localefwd.h:40,
                     from /usr/include/c++/5/string:43,
                     from ./python-rapidjson/rapidjson.cpp:5:
    ./python-rapidjson/rapidjson.cpp: In function ‘void PyInit_rapidjson()’:
    ./python-rapidjson/rapidjson.cpp:867:16: error: return-statement with a value, in function returning 'void' [-fpermissive]
             return NULL;
                    ^
    ./python-rapidjson/rapidjson.cpp:874:31: error: ‘rapidjson_module’ was not declared in this scope
         module = PyModule_Create(&rapidjson_module);
                                   ^
    ./python-rapidjson/rapidjson.cpp:874:47: error: ‘PyModule_Create’ was not declared in this scope
         module = PyModule_Create(&rapidjson_module);
                                                   ^
    In file included from /usr/include/locale.h:28:0,
                     from /usr/include/c++/5/clocale:42,
                     from /usr/include/x86_64-linux-gnu/c++/5/bits/c++locale.h:41,
                     from /usr/include/c++/5/bits/localefwd.h:40,
                     from /usr/include/c++/5/string:43,
                     from ./python-rapidjson/rapidjson.cpp:5:
    ./python-rapidjson/rapidjson.cpp:876:16: error: return-statement with a value, in function returning 'void' [-fpermissive]
             return NULL;
                    ^
    ./python-rapidjson/rapidjson.cpp:888:12: error: return-statement with a value, in function returning 'void' [-fpermissive]
         return module;
                ^
    ./python-rapidjson/rapidjson.cpp: In instantiation of ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode) [with WriterT = rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::ASCII<> >, rapidjson::UTF8<>, rapidjson::ASCII<> >; BufferT = rapidjson::GenericStringBuffer<rapidjson::ASCII<> >; PyObject = _object]’:
    ./python-rapidjson/rapidjson.cpp:821:20:   required from here
    ./python-rapidjson/rapidjson.cpp:513:51: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
                 char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                       ^
    ./python-rapidjson/rapidjson.cpp:571:39: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 char* s = PyUnicode_AsUTF8(object);
                                           ^
    ./python-rapidjson/rapidjson.cpp:607:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:622:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:660:58: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                     PyObject* utcOffset = PyObject_CallMethod(object, "utcoffset", NULL);
                                                              ^
    ./python-rapidjson/rapidjson.cpp:721:90: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                              ^
    ./python-rapidjson/rapidjson.cpp: In instantiation of ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode) [with WriterT = rapidjson::Writer<rapidjson::GenericStringBuffer<rapidjson::UTF8<> > >; BufferT = rapidjson::GenericStringBuffer<rapidjson::UTF8<> >; PyObject = _object]’:
    ./python-rapidjson/rapidjson.cpp:826:20:   required from here
    ./python-rapidjson/rapidjson.cpp:513:51: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
                 char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                       ^
    ./python-rapidjson/rapidjson.cpp:571:39: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 char* s = PyUnicode_AsUTF8(object);
                                           ^
    ./python-rapidjson/rapidjson.cpp:607:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:622:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:660:58: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                     PyObject* utcOffset = PyObject_CallMethod(object, "utcoffset", NULL);
                                                              ^
    ./python-rapidjson/rapidjson.cpp:721:90: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                              ^
    ./python-rapidjson/rapidjson.cpp: In instantiation of ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode) [with WriterT = rapidjson::PrettyWriter<rapidjson::GenericStringBuffer<rapidjson::ASCII<> >, rapidjson::UTF8<>, rapidjson::ASCII<> >; BufferT = rapidjson::GenericStringBuffer<rapidjson::ASCII<> >; PyObject = _object]’:
    ./python-rapidjson/rapidjson.cpp:833:16:   required from here
    ./python-rapidjson/rapidjson.cpp:513:51: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
                 char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                       ^
    ./python-rapidjson/rapidjson.cpp:571:39: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 char* s = PyUnicode_AsUTF8(object);
                                           ^
    ./python-rapidjson/rapidjson.cpp:607:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:622:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:660:58: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                     PyObject* utcOffset = PyObject_CallMethod(object, "utcoffset", NULL);
                                                              ^
    ./python-rapidjson/rapidjson.cpp:721:90: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                              ^
    ./python-rapidjson/rapidjson.cpp: In instantiation of ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode) [with WriterT = rapidjson::PrettyWriter<rapidjson::GenericStringBuffer<rapidjson::UTF8<> > >; BufferT = rapidjson::GenericStringBuffer<rapidjson::UTF8<> >; PyObject = _object]’:
    ./python-rapidjson/rapidjson.cpp:839:16:   required from here
    ./python-rapidjson/rapidjson.cpp:513:51: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
                 char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                       ^
    ./python-rapidjson/rapidjson.cpp:571:39: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 char* s = PyUnicode_AsUTF8(object);
                                           ^
    ./python-rapidjson/rapidjson.cpp:607:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:622:57: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                             char* key_str = PyUnicode_AsUTF8(key);
                                                             ^
    ./python-rapidjson/rapidjson.cpp:660:58: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                     PyObject* utcOffset = PyObject_CallMethod(object, "utcoffset", NULL);
                                                              ^
    ./python-rapidjson/rapidjson.cpp:721:90: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                 PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                              ^
    In file included from ./python-rapidjson/rapidjson.cpp:14:0:
    ./python-rapidjson/docstrings.h: At global scope:
    ./python-rapidjson/docstrings.h:4:20: warning: ‘rapidjson_module_docstring’ defined but not used [-Wunused-variable]
     static const char* rapidjson_module_docstring =
                        ^
    ./python-rapidjson/rapidjson.cpp:846:1: warning: ‘rapidjson_functions’ defined but not used [-Wunused-variable]
     rapidjson_functions[] = {
     ^
    error: command 'gcc' failed with exit status 1

    ----------------------------------------

unix timestamp

Rapidjson supports ISO8601 format for datetime object in dump. Possible to add support for unix timestamp?
(can be handled with "default" parameter in dump but it is nice to have it built-in).

redo benchmarks

after things are somewhat finalized, redo benchmarks. the numbers will have changed.

Should we coerce keys to strings if they are invalid types?

Currently the libraries aren't consistent on this:

>>> import json
>>> json.dumps({True: False})
'{"true": false}'
>>> json.loads(json.dumps({True: False}))
{'true': False}

>>> import ujson
>>> ujson.dumps({True: False})
'{"True":false}'
>>> ujson.loads(ujson.dumps({True: False}))
{'True': False}
>>>

But rapidjson raises a TypeError:

>>> import rapidjson
>>> rapidjson.dumps({True: False})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: keys must be a string

I'm kind of ok with that but wondering how compatible we want to be

indent param

if a string is passed to indent, the indent char is the first char of the string and the indent char count is the length of the indent string. e.g. indent="\t\t" would be 2 tabs (but so would indent="\tA".

decide whether to keep this. might be nice if someone wanted to use tabs instead of spaces.

if kept, need to raise an exception if the string is not ascii.

[versioning] use minor version and free patch version for minor fixes

This is just a suggestion regarding versioning. Given {major}.{minor}.{patch}, it may be worth considering using the minor version for releases, and thus freeing the patch version for hot fixes. For instance, having a release 0.1.0 allows for multiple very minor patches/fixes: 0.1.1, 0.1.2, 0.1.3, etc.

Segfaults and wrong parse errors when using object_hook

First, I am not really sure about my diagnosis because it is not easy to reproduce, but I have ran tests often and long enough to be quite sure. It seems like I'm seeing segfaults or irreproducible (i.e. when feeding the json again, it parses just fine) parse errors ("ValueError: Parse error at offset 2881568: Terminate parsing due to Handler error.") when using rapidjson.loads with an object_hook callback.

Using Python's faulthandler, the segfaults come from no definite place. The offset the parse error happens is always on the "}," between two records (so after the object_hook callback has been called?).

I tried this with Alpine Linux 3.4.4 and their Python 3.5 apks, a freshly compiled Python 3.5.2, and also a freshly compiled Python 3.5.2 on latest Debian. python-rapidjson is 0.0.6 from PyPI.

My application is a AWS Kinesis client app, which just parses jsons from the incoming stream and writes these to S3. I cannot reproduce without writing to S3, so it seems like boto3/urllib3/socket interaction is required for this to happen. Chances to see this increase when under load (continuously parsing ~9MB jsons) and using multiple threads, although I experienced these faults also completely single threaded.

Automatically deploy sdist and wheel on PyPI

It would be nice to configure Travis CI to automatically upload sdist and binary wheel to PyPI.

Floating point number discrepancy

See #8

datetime mode

currently, there is 'none' and 'iso8601'. iso8601 should include the time zone. might want another mode that ignores the time zone as well. also need to expose the constants in python.

Class inheritance seems weird

Maybe not a bug, but I found something odd. The following custom class cannot be used:

from rapidjson import Encoder

class MyEncoder(Encoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

This results in the following error:

>>> MyEncoder(sort_keys=True)
TypeError: object.__init__() takes no parameters

Does this make sense? Or is this an indicator that the Encoder class is maybe not set up correctly?

In any case, I'd be very interested to learn why this happens :)

Enable the possibility to not bundle rapidjson with the package?

Both rapidjson and python-rapidjson have been bundled for conda-forge,

However, the latter makes use of the rapidjson submodule.

It would probably make sense for the setup.py to handle the case where an installation prefix for rapidjson is provided instead of a submodule.

For reference, such a setup.py has been done for pyzmq which can either bundle zeromq or not.

What about using upstream RapidJSON 1.1?

Hi Ken,

I spent some hours "rebasing" py-rapidjson on top of current RapidJSON version 1.1, released a few days ago.

The branch currently working ok, all tests pass; also, with a grain of salt, after a few iteration of the benchmarks, it seems almost always perform slightly better than current master. Where it is slower, its mainly caused by a few other impacting (hopefully) changes, that I may either try to smooth, or extract in a different branch, or forget about, at your option:

does not use native C++ "numbers" (ie floats and ints), but rather use the "raw value", to parse them thru {PyFloat|PyDecimal|PyLong}_FromString(): for this reason, I dropped the option "precise_floats"
I did not (yet?) rebased/cherrypicked your str optimizations to the writer you made here and here
I used a submodule to bring in upstream RapidJSON sources, which, expecially if you bless me on doing point 2 above, seems a better/cleaner solution rather than including just the sources as it was; it's of course up to you, I can just easily use subtree to include it while keeping its history/reference with the upstream sources, or simply copy version 1.1 upstream, like before, should you prefer that

Current state is in my rj-v1.1-as-submodule.

Am I on the right path? Could any parts of that branch be accepted as a PR?

Thank you!

allow_nan on loads?

allow_nan is a param on dumps. should it be a param on loads too?

Error installing on ubuntu

Hi, I'm trying to install this on ubuntu. However, getting errors:
I have g++ version 5.4

...
...
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/python-rapidjson
  gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./rapidjson/include -I/home/ubuntu/anaconda2/include/python2.7 -c ./python-rapidjson/rapidjson.cpp -o build/temp.linux-x86_64-2.7/./python-rapidjson/rapidjson.o
  cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::NaN()’:
  ./python-rapidjson/rapidjson.cpp:232:43: error: too few arguments to function ‘PyObject* PyFloat_FromString(PyObject*, char**)’
               value = PyFloat_FromString(str);
                                             ^
  In file included from /home/ubuntu/anaconda2/include/python2.7/Python.h:89:0,
                   from ./python-rapidjson/rapidjson.cpp:1:
  /home/ubuntu/anaconda2/include/python2.7/floatobject.h:48:24: note: declared here
   PyAPI_FUNC(PyObject *) PyFloat_FromString(PyObject*, char** junk);
                          ^
  ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::Infinity(bool)’:
  ./python-rapidjson/rapidjson.cpp:262:43: error: too few arguments to function ‘PyObject* PyFloat_FromString(PyObject*, char**)’
               value = PyFloat_FromString(str);
                                             ^
  In file included from /home/ubuntu/anaconda2/include/python2.7/Python.h:89:0,
                   from ./python-rapidjson/rapidjson.cpp:1:
  /home/ubuntu/anaconda2/include/python2.7/floatobject.h:48:24: note: declared here
   PyAPI_FUNC(PyObject *) PyFloat_FromString(PyObject*, char** junk);
                          ^
  ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::RawNumber(const char*, rapidjson::SizeType, bool)’:
  ./python-rapidjson/rapidjson.cpp:334:49: error: too few arguments to function ‘PyObject* PyFloat_FromString(PyObject*, char**)’
                   value = PyFloat_FromString(pystr);
                                                   ^
  In file included from /home/ubuntu/anaconda2/include/python2.7/Python.h:89:0,
                   from ./python-rapidjson/rapidjson.cpp:1:
  /home/ubuntu/anaconda2/include/python2.7/floatobject.h:48:24: note: declared here
   PyAPI_FUNC(PyObject *) PyFloat_FromString(PyObject*, char** junk);
                          ^
  ./python-rapidjson/rapidjson.cpp: In member function ‘bool PyHandler::HandleIso8601(const char*, rapidjson::SizeType)’:
  ./python-rapidjson/rapidjson.cpp:558:89: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                                                                     rapidjson_timezone_utc);
                                                                                           ^
  ./python-rapidjson/rapidjson.cpp:558:89: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
  ./python-rapidjson/rapidjson.cpp:642:89: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
                                                                     rapidjson_timezone_utc);
                                                                                           ^
  ./python-rapidjson/rapidjson.cpp:642:89: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
  ./python-rapidjson/rapidjson.cpp: In function ‘PyObject* rapidjson_loads(PyObject*, PyObject*, PyObject*)’:
  ./python-rapidjson/rapidjson.cpp:763:66: error: ‘PyUnicode_AsUTF8AndSize’ was not declared in this scope
           jsonStr = PyUnicode_AsUTF8AndSize(jsonObject, &jsonStrLen);
                                                                    ^
  ./python-rapidjson/rapidjson.cpp:816:47: error: ‘PyUnicode_AsUTF8’ was not declared in this scope
                   emsg = PyUnicode_AsUTF8(evalue);
                                                 ^
  ./python-rapidjson/rapidjson.cpp: In function ‘PyObject* rapidjson_dumps_internal(WriterT*, BufferT*, PyObject*, int, int, PyObject*, int, int, unsigned int, DatetimeMode, UuidMode)’:
  ./python-rapidjson/rapidjson.cpp:923:68: error: there are no arguments to ‘PyUnicode_AsUTF8AndSize’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8AndSize’ must be available [-fpermissive]
               char* decStr = PyUnicode_AsUTF8AndSize(decStrObj, &size);
                                                                      ^
  ./python-rapidjson/rapidjson.cpp:923:68: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
  ./python-rapidjson/rapidjson.cpp:938:68: error: there are no arguments to ‘PyUnicode_AsUTF8AndSize’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8AndSize’ must be available [-fpermissive]
               char* intStr = PyUnicode_AsUTF8AndSize(intStrObj, &size);
                                                                      ^
  ./python-rapidjson/rapidjson.cpp:983:57: error: there are no arguments to ‘PyUnicode_AsUTF8AndSize’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8AndSize’ must be available [-fpermissive]
               char* s = PyUnicode_AsUTF8AndSize(object, &l);
                                                           ^
  ./python-rapidjson/rapidjson.cpp:1019:61: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                           char* key_str = PyUnicode_AsUTF8(key);
                                                               ^
  ./python-rapidjson/rapidjson.cpp:1034:61: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
                           char* key_str = PyUnicode_AsUTF8(key);
                                                               ^
  ./python-rapidjson/rapidjson.cpp:1195:95: error: there are no arguments to ‘PyUnicode_AsUTF8’ that depend on a template parameter, so a declaration of ‘PyUnicode_AsUTF8’ must be available [-fpermissive]
               PyErr_Format(PyExc_TypeError, "%s is not JSON serializable", PyUnicode_AsUTF8(repr));
                                                                                                 ^
  ./python-rapidjson/rapidjson.cpp: At global scope:
  ./python-rapidjson/rapidjson.cpp:1343:8: error: ‘PyModuleDef’ does not name a type
   static PyModuleDef rapidjson_module = {
          ^
  In file included from /usr/include/locale.h:28:0,
                   from /usr/include/c++/5/clocale:42,
...
....

   In file included from ./python-rapidjson/rapidjson.cpp:1:0:
    ./python-rapidjson/docstrings.h: At global scope:
    ./python-rapidjson/docstrings.h:4:14: warning: ‘rapidjson_module_docstring’ defined but not used [-Wunused-variable]
     PyDoc_STRVAR(rapidjson_module_docstring,
                  ^
    /home/ubuntu/anaconda2/include/python2.7/Python.h:170:37: note: in definition of macro ‘PyDoc_VAR’
     #define PyDoc_VAR(name) static char name[]
                                         ^
    ./python-rapidjson/docstrings.h:4:1: note: in expansion of macro ‘PyDoc_STRVAR’
     PyDoc_STRVAR(rapidjson_module_docstring,
     ^
    ./python-rapidjson/rapidjson.cpp:1337:1: warning: ‘rapidjson_functions’ defined but not used [-Wunused-variable]
     rapidjson_functions[] = {
     ^
    error: command 'gcc' failed with exit status 1

Null-bytes in dictionary keys

Follows up #38

The issue persists for dictionary keys:

(Pdb) import json
(Pdb) json.dumps({'\x00': 'a'})
'{"\\u0000": "a"}'

(Pdb) import rapidjson
(Pdb) rapidjson.dumps({'\x00': 'a'})
'{"":"a"}'

Check for circular references

See #6

Generalize support for lists and tuples to extend to iterators

It would be great to be able to serialize, for example, a generator as that can be more efficient in terms of memory consumption; instead of having to construct a list of N items prior to serializing them, python-rapidjson could consume items from a generator and produce a JSON list.

Relaxed JSON syntax from rapidjson-1.1

Id like to be able to use the relaxed json syntax optionally allowed in rapidjson-1.1. (See http://rapidjson.org/md_doc_dom.html)

Specifically
kParseTrailingCommasFlag
kParseCommentsFlag

Im not worried about roundtrip read/write, but at this point just for read.

I think this just needs a blessed parameter exposed to allow for these to be enabled at the C lib.

Unable to compile MacOS wheel for Python 3.4

Under MacOS/clang, the compilation of the module with Python 3.4 runtime fails, see https://travis-ci.org/python-rapidjson/python-rapidjson/jobs/273856525

Request for opinion: different way to specify related options

While (re)implementing some features I'd like to propose for acceptance (native numbers and issue #61 for example), it occurred to me that there is some kind of dichotomy in the way they are activated by passing different arguments to the exposed API functions.

Since I'm going to work toward a 0.1 release, it seems the right time to unify the two styles, assuming there is some consensus. Since I'm obviously biased, I'm asking your opinion here.

One style, inherited by the standard library json module, uses a set of boolean options to enable/disable different behaviours: for example, we have allow_nan to enable NaN recognition, and use_decimal to prefer decimal.Decimal over plain floats.

The other style, which I used to implement the support for datetime objects, uses a single option datetime_mode that takes a combination of bit flags to select a particular serialization format and different behaviours wrt timezones. This allowed me to restrict the number of options accepted by the API.

An example of the former is:

>>> print(dumps(foo, allow_nan=False, use_decimal=True)

while the latter looks like:

>>> print(dumps(foo, datetime_mode=DM_ISO8601 | DM_NAIVE_IS_UTC)

To implement native numbers (that is, to tell underlying RapidJSON to use architecture native numbers, a much faster codepath than the infinite precision one), I could code it either as:

>>> result = loads(json, native_numbers=True, use_decimal=True)

or like:

>>> result = loads(json, number_mode=NM_NATIVE_NUMBERS | NM_DECIMAL_FLOATS)

(the focus being on the bitset number_mode option, not on flag's names).

What do you think? Which style would you prefer?

If standalone options is considered better, should I (re²)implement datetime handling in that style?
If instead we go the bitset way, should I remove/deprecate the old options or leave them untouched?

Thank you for any feedback!