How about adding a feature for pretty output?

When you do:

Json.object().at("count", 10)

will automatically figure that there is not property named "count" in the object and add one with the value 10. The 2nd argument here is intended to be the default for missing properties. It seemed sensible for that accessor 'at' method to mutate the object because from a declarative perspective, a default value is a default value, always the same no matter the user. But I find myself worrying over modifying existing objects, especially if they are being saved back into a database where you want to avoid storing default to keep the data sparse and less storage consuming. The solution is call 'dup()' so that the original DB entity is not modified. But that's not very pleasant either. On the other hand, I've very rarely come across a use case where I really wanted 'at' the insert the specified default. So, breaking backward compatibility unfortunately, the behavior should be change for 'at' not the modify the object.

Hi,
Found I can get a huge increase in speed by caching in the escaper. In my case I am outputting periodic updates to the same data, so its especially useful. the escape method was using 5% CPU, now its below 1%. I know you dont want any dependencies but I used Google guava cache. Code is here if you want to use it:

escapeHash = CacheBuilder.newBuilder()
                   .maximumSize(1000)
                   .build(
                           new CacheLoader<CharSequence, String>() {
                             public String load(CharSequence plainText) throws Exception {
                                 StringBuilder escapedString = new StringBuilder(plainText.length() + 20);
                                try {
                                  escapeJsonString(plainText, escapedString);
                                } catch (IOException e) {
                                  throw new RuntimeException(e);
                                }
                                return escapedString.toString();
                             }
                           });
      }

      public String escapeJsonString(CharSequence plainText) {
        return escapeHash.getUnchecked(plainText);
      }

The current default implementation of Json.toString will output an ellipsis (...) when the traversal is about to output the same object (same Java reference). This is to avoid circularity issues and because the aim of toString was for display purposes. We might break backward compatibility (and maybe this can be made optional), but this decision is a bit counter-intuitive. One expected toString to at least produce a valid JSON string. Because we set out to support graphs in this API and not just trees, we have to handle circular structures. But we can do this in a separate method, e.g. displayString or some such. Alternatively, we could offer a hook to be provided by the user to deal with serializing the same object, but in a different context.

This should be in a related to mJson project (e.g. mJsonPlus that could contains all extra goodies while keeping mJson itself small). The idea is to create Java annotations that validate input and output of Json methods:

@JsonValidate("/resources/schemas/person.json")
public void doSomethingWithPerson(Json person)
{
....// here we know the person arguments is a valid person
}

such validating annotations would be placed only at entry points...of course, to avoid validating every single time something as passed as an argument. Alternatively, we might have some meta information kept in a global map that says that such and such objects has been validated and follows a schema.

I integrated mjson inside a JSF web application and it requires the Json instances to be serializable:
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: mjson.Json$ObjectJson

The issue may be solved by adding "implements Serializable" to the objects ArrayJson, BooleanJson, NullJson, NumberJson, ObjectJson and StringJson. What do you think?

Persistable json objects usually carry some kind of technical identifier with them. Is there some way to compare two json objects without technical properties?

For example:
{"id":"1","type":"car","model":"Ferrari","color":"red"}
{"id":"2","type":"car","model":"Ferrari","color":"red"}

Both objects represent the same car (functional equality) but they are not equal/identical because of their different IDs. If there would be a way to exclude/ignore certain properties during the equals operation would be a great feature. Maybe the API could look something like car1.functionalEquals(car2,"id"); whereas the last argument is a variable parameter list. Or maybe it would be better to mark technical properties within the interal data structure of mjson itself for beeing able to handle nested/more complex objects. This would require some API for marking properties as beeing "excluded" from the functional equals comparison.

For a predictable standard string serialization, we should use RFC 7159

There is the occasional parsing bug that manifests itself. Also, our parser is not written for super high performance so we may decide to replace it some day. For that, we need tests that JSON documents & values are parsed correctly into the corresponding Json elements.

The unit tests should focus especially on corner cases with lots of ill-formed JSON documents. Well-formed document with no syntax errors are important as well, but new bugs are likely to be discovered with how the parser deals with bad syntax.

Also, it would be nice, once we have error test cases to improve the error reporting of the parser itself..

This seems like a new effort, but important for REST services to have something like this: http://jsonpatch.com/, so we could support it.

Seems pretty advance and nice:

https://code.google.com/p/json-path/

Since Spring framework is adopting it, probably it's going to gain industry acceptance, and there's nothing else like it.

1. I have my schema file in src/main/resources
2. In schema i have construction like "$ref": "#/definitions/deleteSchema"
3. I open schema
   ResourceLoader resourceLoader ;
   ....
   Json.Schema schema = Json.schema(resourceLoader.getResource("classpath:" + jsonSchemaFile).getURI());
4. I have NPE

Caused by: java.lang.RuntimeException: java.lang.NullPointerException
        at mjson.Json$DefaultSchema.<init>(Json.java:1009) ~[mjson-1.3.jar:na]
        at mjson.Json.schema(Json.java:1035) ~[mjson-1.3.jar:na]
        ... 48 common frames omitted
Caused by: java.lang.NullPointerException: null
        at mjson.Json.resolveRef(Json.java:462) ~[mjson-1.3.jar:na]
        at mjson.Json.expandReferences(Json.java:508) ~[mjson-1.3.jar:na]
        at mjson.Json.expandReferences(Json.java:529) ~[mjson-1.3.jar:na]
        at mjson.Json.expandReferences(Json.java:519) ~[mjson-1.3.jar:na]
        at mjson.Json$DefaultSchema.<init>(Json.java:1006) ~[mjson-1.3.jar:na]

How would you implement firing an event whenever data in the Json object changed? I have a use case where I want to trigger external processing when certain values are added/updated, or deleted.

Hi,

I'm trying to convert JSON from/to String as follows:

    String myJSONString = "{\"user\":\"myuser\",\"pw\":\"mypass\",\"log\":true}";
    System.out.println(myJSONString);
    Json json = Json.read(myJSONString);
    System.out.println(json.toString());

That is, I expect that myJSONString.equals(json.toString()) returns true, but it does not.

The values that System.out.println() show are:

{"user":"myuser","pw":"mypass","log":true}
{"log":true,"pw":"mypass","user":"myuser"}

For myJSONString and json.toString(), respectively. Therefore, json.toString() inverts the string. Is this a bug? I want to load/store the same JSON string from/to disk several times but due to this behaviour I'm not able to do that.

PS: The problem is that next time I load the MyJSONString from this it cannot be validated using the JSON schema.

Nice work on mJson, I like the concept. Been trying it out and I found I added these pretty quick for big trees. Typing at("person").at(".address").at("city")... gets quite tedious :-)

/**
* Get the Json node at "person.address.city"
* @param path
* @return
/
public Json atPath(String path){
String[] paths = path.split(".");
return atPath(paths);
}
/*
* Get the json node at {"person","address","city"}
* Convenient when using CONSTANTS, atPath(PERSON,ADDRESS,CITY)
* @param path
* @return
*/
public Json atPath(String ... path ){
Json json=null;
for(String k:path){
if(json==null)json=at(k);
json=json.at(k);
if(json==null) return null;
}
return json;
}

You are creating a topnull object in Json$NullJson class.
static NullJson topnull = new NullJson();
This instance is static and therefore, will never be garbage collected.
Every time some json data containing null fields is processed, the enclosing field of this object is growing.

You can reproduce the problem running this code:

package mjson;

import java.io.File;
import java.io.FileNotFoundException;
import java.util.Scanner;
import java.util.stream.IntStream;

public class Main {
	public static void main(String[] args) throws FileNotFoundException, InterruptedException {
		String content = new Scanner(new File("resources/file.json")).useDelimiter("\\Z").next();
		Scanner in = new Scanner(System.in);
	
		while(true){
			in.nextLine();
			IntStream.range(0, 100000).forEach(i -> {
				Json.read(content);
			});	
			System.out.println("SIZE: "+ Json.NullJson.topnull.enclosing.asList().size());
		}
	}
}

Used file.json:

{
    "whatever1": null,
    "whatever2": null,
    "whatever3": null,
    "whatever4": null,
    "whatever5": null,
    "whatever6": null,
    "whatever7": null,
    "whatever8": null,
    "whatever9": null
}

On the screenshot below, you can see what happens on the heap.
I sequentially run multiple Json.read() calls and then trigger GC.
After just few runs, heap size is already above 0.5GB.

Quick fix:
With @KamilKrol we introduced a quick fix, by removing the topnull field and creating new NullJson instance every time DefaultFactory#nil() is called.
This seems to be solving the leak problem, but the proper solution probably requires investigating why NullJson#enclosing field is not empty after processing is finished.

The following code produces a NullPointerException:

Json json = Json.object();
String value = json.at("notAvailable").asString();

What is the advantage of json.at("...") returning null instead of a json object? I would have expected the java null value from the asString() method.

The method Json.read() for example will never return the actual Java null. Could this also be done for the method json.at("...")?

I have several cases where I have the json element in a hierarchy (eg after an event has occurred) but I need to know its key in the parent json. Basically I need to recurse the path back to the top level. This is remarkably hard to do efficiently! Ive considered several possibilities:

1. Use google guava BiMap instead of HashMap in the ObjectJson. Then use .inverse() to find the key. Would work but BiMap must have unique values (a Set), and I suspect that will cause trouble
2. store the key in the child object when the child is created. Also works but needs care or it causes bugs when serialising or when making misc changes like moving parent.
3. store a ref to the parent key in the child somehow - much like 2)
4. iterate the keys in the parent looking for an equals() match on the associated value - nice, but is it efficient in a big parent? What about two separate keys with the same value? - the BiMap problem.

Any ideas?

The API is simple and well-defined, but our pluggable factory allows for different implementations of all JSON types. It would be nice to have a comprehensive test harness so alternative implementations can be tests. One way to organize the test suite is by JSON type. So have a class containing tests for boolean, another for strings, another for objects, another for arrays, another for numbers and one for nulls. Because of the unified interface for all JSON types, it should be easy to just go over the methods and make sure they all work.

This example:

{
"type":"something",
"value1":"Blabla",
"value2":"years"
"value3":"wtf"
}

is missing a coma after the 3d value, and instead of an error being thrown it parses as:

{"value2":"wtf","value1":"Blabla","type":"something"}

Hey , is it possible to pretty print Json as a string ? For now I found only 2 toString methods(default one and one with maxCharacters) and non of them is able to pretty print Json object

Indeed, the currently existing test-case actually runs into infinite loop with them in (testSortedArrayMerge).

Possible fix (at least the testcase is fixed by that):
unserializable@63d1773?diff=unified

Stack overflow error caused by mjson parsing of untrusted JSON String

Description

Using mjson to parse untrusted JSON String may be vulnerable to denial of service (DOS) attacks. If the parser is running on user supplied input, an attacker may supply content that causes the parser to crash by stackoverflow.

Error Log

Exception in thread "main" java.lang.StackOverflowError
	at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:165)
	at mjson.Json.factory(Json.java:1192)
	at mjson.Json.array(Json.java:1291)
	at mjson.Json$Reader.readArray(Json.java:2787)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)
	at mjson.Json$Reader.read(Json.java:2715)
	at mjson.Json$Reader.readArray(Json.java:2788)

PoC

        <dependency>
            <groupId>org.sharegov</groupId>
            <artifactId>mjson</artifactId>
            <version>1.4.1</version>
        </dependency>

import mjson.Json;

public class PoC {

    public final static int TOO_DEEP_NESTING = 9999;
    public final static String TOO_DEEP_DOC = _nestedDoc(TOO_DEEP_NESTING, "[ ", "] ", "0");


    public static String _nestedDoc(int nesting, String open, String close, String content) {
        StringBuilder sb = new StringBuilder(nesting * (open.length() + close.length()));
        for (int i = 0; i < nesting; ++i) {
            sb.append(open);
            if ((i & 31) == 0) {
                sb.append("\n");
            }
        }
        sb.append("\n").append(content).append("\n");
        for (int i = 0; i < nesting; ++i) {
            sb.append(close);
            if ((i & 31) == 0) {
                sb.append("\n");
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        String jsonString = TOO_DEEP_DOC;
        Json.read(jsonString);
    }
}

Rectification Solution

1. Refer to the solution of jackson-databind: Add the depth variable to record the current parsing depth. If the parsing depth exceeds a certain threshold, an exception is thrown. (FasterXML/jackson-databind@fcfc499)

2. Refer to the GSON solution: Change the recursive processing on deeply nested arrays or JSON objects to stack+iteration processing.(（google/gson@2d01d6a20f39881c692977564c1ea591d9f39027）)

		public Json delAt(String property) 
		{
			Json el = object.remove(property);
			removeParent(el, this);
			return this;
		}

when there is no value for the property being deleted, the removeParent call leads to an NPE

Parsing currently assumes we have the full document and it's a one short operation. But for processing streams of Json documents a user of mjson needs to do extra work to separate the start and end of documents which amounts almost to a full parsing as well. Instead, we should have a Stream<Json> readStream(source) method that does that automatically.

Hi,
do you plan to ad date support?

Regards
Massi

The interface will of course fatten a bit, but that's also the point of having a single class. Instead of writing

object.at(propertyname).asString()

we could write

object.asString(propertyname)

or for arrays:

object.asString(10)

and like this for all asXXX methods.

Or drop the "as" here since we are not "at the value" yet to be say that we look at it "as something". So maybe just like this:

object.string(propname)
object.bool(properyname)

that's even shorter, but the problem is new method names and less consistency with the rest of the API. Is saving a few characters worth the loss of consistency? Don't think so..

As stated in the javadoc of the interface Json.Factory the method make(Object) allows me to plug-in my own mapping of java.util.Date to Json. But how to achive this without reimplementing the whole method?

Futhermore I would like to know if it is possible to globally map the other way around from a Json string to java.util.Date. I would have expected something like Json.asDate() or Json.asType(Class type) for the conversion to Java instances.

Stack overflow error caused by mjson parsing of untrusted JSON String

Description

Using mjson to parse untrusted JSON String may be vulnerable to denial of service (DOS) attacks. If the parser is running on user supplied input, an attacker may supply content that causes the parser to crash by stackoverflow.

Error Log

Exception in thread "main" java.lang.StackOverflowError
	at mjson.Json$Reader.next(Json.java:2630)
	at mjson.Json$Reader.skipWhiteSpace(Json.java:2669)
	at mjson.Json$Reader.read(Json.java:2709)
	at mjson.Json$Reader.readObjectKey(Json.java:2751)
	at mjson.Json$Reader.readObject(Json.java:2765)
	at mjson.Json$Reader.read(Json.java:2718)
	at mjson.Json$Reader.readObjectKey(Json.java:2751)
	at mjson.Json$Reader.readObject(Json.java:2765)
	at mjson.Json$Reader.read(Json.java:2718)
	at mjson.Json$Reader.readObjectKey(Json.java:2751)
	at mjson.Json$Reader.readObject(Json.java:2765)
	at mjson.Json$Reader.read(Json.java:2718)
	at mjson.Json$Reader.readObjectKey(Json.java:2751)
	at mjson.Json$Reader.readObject(Json.java:2765)
	at mjson.Json$Reader.read(Json.java:2718)
	at mjson.Json$Reader.readObjectKey(Json.java:2751)
	at mjson.Json$Reader.readObject(Json.java:2765)
	at mjson.Json$Reader.read(Json.java:2718)

PoC

        <dependency>
            <groupId>org.sharegov</groupId>
            <artifactId>mjson</artifactId>
            <version>1.4.1</version>
        </dependency>

import mjson.Json;

public class PoC {

    public final static int TOO_DEEP_NESTING = 9999;
    public final static String TOO_DEEP_Object = _nestedDoc(TOO_DEEP_NESTING, "{ ", "} ", "0");


    public static String _nestedDoc(int nesting, String open, String close, String content) {
        StringBuilder sb = new StringBuilder(nesting * (open.length() + close.length()));
        for (int i = 0; i < nesting; ++i) {
            sb.append(open);
            if ((i & 31) == 0) {
                sb.append("\n");
            }
        }
        sb.append("\n").append(content).append("\n");
        for (int i = 0; i < nesting; ++i) {
            sb.append(close);
            if ((i & 31) == 0) {
                sb.append("\n");
            }
        }
        return sb.toString();
    }

    public static void main(String[] args) {
        String jsonString = TOO_DEEP_Object;
        Json.read(jsonString);
    }
}

Rectification Solution

1. Refer to the solution of jackson-databind: Add the depth variable to record the current parsing depth. If the parsing depth exceeds a certain threshold, an exception is thrown. (FasterXML/jackson-databind@fcfc499)

2. Refer to the GSON solution: Change the recursive processing on deeply nested arrays or JSON objects to stack+iteration processing.(（google/gson@2d01d6a20f39881c692977564c1ea591d9f39027）)

Not sure if a great idea, but an 'overlay' or 'overwrite' method would behave sort of like 'with', except recursively it will perform a merge (actually maybe 'merge' is a better name for such a method then): if A is overlaid over B, all properties in A that are not in B will be copied to B and also all properties in A that are also in B will replace B's values. But because this is done recursively, we may have nested properties deep in B's structure be preserved instead of just disappearing.

So this is essentially a recursive "with". While "with" overwrite blindly, a recursive with does a deep merging.

I've only seen the need for this popup very rarely though.

The current release of MJSON (1.4.1) includes junit as a dependency in it's pom. This is surprising since it promises to have no dependencies. I see that it's actually been fixed in master but no new version has been released since 2017.

Since we moved the test cases to junit (from testng), the suite of JSON schema tests doesn't run anymore. This needs fixing before 1.4.

v1.4.0 binary seems to be compiled with Java 7 (Unsupported major.minor version 51.0)
Is there a binary version for Java 1.6?

Thanks!