square / moshi Goto Github PK

This is a tracking issue for the slight improvement that can be made whenever Okio 1.4 is released.

JsonWriter's string method has two calls to sink.writeUtf8(string.substring(#, #)) which will be able to be replaced by sink.writeUtf8(string, #, #). This avoids the String & backing char[] allocation as well as the data copy to only read it back out again for encoding.

I have a project where it's necessary to be able to modify a client-side object based on a JSON keypath/value pair that's sent over a persistent websocket. Is this something that's on the roadmap with Moshi?

Use case:

Say we have (excuse the useless example)

{
"kitchen": {
    "fridge": {
        "apple": {
            "status": "ok"
        },
        "pear": {
            "status": "ok"
        },
        "curry": {
            "status": "ok"
        }
    },
    "pantry": {
        "biscuit": {
            "status": "ok"
        },
        "chocolate": {
            "status": "ok"
        }
    }
}

}

At some point, the server says that the apples have gone out of date. Instead of sending over the whole kitchen, it'll do something like :

{"kitchen.fridge.apple.status": "moldy"}

Based on the server sending that over, the client side kitchen should be updated to have a moldy apple.

Here's another explanation & accompanying library (which doesn't actually work out of the box).
http://www.arg3.com/blog/2014/02/20/java-key-path-json-objects/

I modified that library a little, and have cobbled together an ugly working solution, but essentially it works by

*. Parse (with Gson) my object into JSONObject kitchen.
*. Do kitchen.putKeyPath("fridge.apple.status", "moldy")
*. Parse the kitchen back into it's proper class with Gson

It feels very wrong, but I haven't been able to think of anything cleaner.

Is this something within the same problem domain as Moshi, or should it be a more standalone thing?

Would be handy for development to detect integration problems.

In ClassFactory.get(), it uses reflection to get method ObjectStreamClass.getConstructorId(). However, the method is not available in Android 2.3. Is there any workaround? Or should there be at least some documentation for this?

This JSON
{
"id":"2",
"name":"glpi",
"firstname":null
}

fails: Expected a value but was NULL at path $.firstname

class ObjectJsonAdapter in file StandardJsonAdapters.java
has switch (reader.peek()) {}
there are BEGIN_ARRAY, BEGIN_OBJECT, STRING, NUMBER, BOOLEAN
and noooooo NULL value among cases !!!

Might need to change our APIs to let the user provide a BufferedSource, since right now we might consume more data than we should.

#33

Is there a configuration to make moshi work with Proguard? specifically the @FromJson and ToJson annotation?

I figured if I put this:

-keep class com.squareup.moshi.** { *; }
-keepattributes *Annotation*

it might work but I get Expected at least one @ToJson or @FromJson method on com.example.app.mypackage. Since it works perfectly when proguard is off, I'm guessing my configuration is wrong. Thanks in advance :)

I would like to have the ability to convert something like
List myModelList = new ArrayList();
To JSON. The easiest way I can find and the GSON kind of way to do it is:
moshi.adapter(Types.newParameterizedType(List.class, MyModel.class);

The problem is the Types package is private. Am I missing something or can we make this package or method public so I can easily convert a list to JSON.

https://tools.ietf.org/html/rfc6901

Any thoughts on support JsonPointer for self-referencing values in a JSON doc? JSR-367 is planning to support it.

Currently https://github.com/square/moshi#borrows-from-gson can make it seem like we have no @SerializedName analogous.

Some people are very concerned about speed... especially when it comes to JSON parsing and reflection. So my question would be:

Does it make sense for you guys to add a parser generator on top (something like ig-json-parser and LoganSquare)?

https://github.com/square/moshi/blob/master/moshi/src/main/java/com/squareup/moshi/JsonReader.java#L1227

 // Skip a # hash end-of-line comment. The JSON RFC doesn't specify this behaviour, but it's
 // required to parse existing documents. See http://b/2571423.

The URL is invalid

JDK 8 defines ISO_DATE_TIME format, and we should consider using that as the representation of Date/Time objects. JSR-367 is standardizing on this format as well.

Copy script from other project. Set up empty gh-pages branch before running.

It’d be neat if JsonReader/JsonWriter could write to a virtual DOM rather than a stream.

https://twitter.com/MoshiFrenchie

This may sound like a stupid request, and no other (know to me) json library allows such possibilities, but there is a case when this is very handy. I have the following setup:

• One api client that uses Moshi for json parsing.
• The api client is shipped with predefined pojos, some of which require custom adapter factories (set by the api client).
• The api client is used by multiple teams (not only onsite).
• Other teams consume other "private" apis. Which result in -> new pojos and eventually requirement to set new adapters/adapter factories.

The point is that the client could expose in it's builder methods that will just mirror the ones in Moshi.Builder (currently 4 of them), but a more cleaner solution, would be just have one method addAll(builder) or addFrom(builder).

I'm trying to parse a a list but I can't find docs on how to do that. I already made my custom adapter class, but when I do the following:

This doesn't work with lists

JsonAdapter<MyData> adapter = moshi.adapter(MyData.class)

This doesn't work

JsonAdapter<List> adapter = moshi.adapter(List.class)

How do I parse a List<MyData> ?

When I use .nullSafe(), I get this error: Expected at least one @toJson or @FromJSON method on com.squareup.moshi.JsonAdapter$1
I use Moshi 1.0.0

I try to use Moshi in my Kotlin project which needs to parse a heterogeneous data as strings: it can be values like 5stars, something, true - and I need to parse them all as string. But when Moshi sees true it tries to parse it as boolean and throws an exception when parsing 3rd item:

com.squareup.moshi.JsonDataException: Expected a string but was BOOLEAN at path $.Data.FilterDictionary[5].Items[0].Options[1].Value

So I tried to come up with custom adapter and @JsonQualifier:

data class DictionaryEntry(
  val Options: List<Property>
)

data class Property(
  val Name: String,
  @ForceToString
  val Value: String
)

@Retention(AnnotationRetention.RUNTIME)
@JsonQualifier
annotation class ForceToString

class ForceToStringAdapter {
  @ToJson
  fun toJson(@ForceToString value: String): String = value
  @FromJson @ForceToString
  fun fromJson(value: String): String = throw UnsupportedOperationException()
//  fun fromJson(value: String): String = value
}

I register it:

    val moshi = Moshi.Builder()
      .add(ForceToStringAdapter())
      .build()
    val retrofit = Retrofit.Builder()
      .addCallAdapterFactory(RxJavaCallAdapterFactory.create())
      .addConverterFactory(MoshiConverterFactory.create(moshi))
      .client(client)
      .baseUrl("...")
      .build()

But alas, nothing happens - same error about Boolean and no UnsupportedOperationException() gets thrown as I would expect from the above code...

I don't know if this is some library error or if I am doing something wrong or If this is something Kotlin specific.

Gson and Moshi differ in their deserialization of null to primitive types:

• Gson allows null for a primitive type, like int. It will result in 0.
• Moshi is strict in this way and will throw an exception.

While I understand that the model should be Integer in cases where null is expected/allowed, this difference in both libraries makes the migration difficult.

Do you plan to allow null for primitive types? Maybe not by default but as an option on Moshi.Builder? If not this difference should at least be mentioned in the README.md.

Given

class AnInterfaceAdapter {
  @ToJson String to(AnInterface ai) {
    return "an_interface";
  }
}

Moshi moshi = new Moshi.Builder()
    .add(new AnInterfaceAdapter())
    .build();
moshi.adapter(AnInterface.class);

An error message is thrown saying no adapter could be found for AnInterface. This is not strictly true and misleading, the problem is that only a "to" adapter could be found but no "from" adapter. We should report a better error message.

To exclude a field from class serialization I create a custom type adapter to write all fields except it, but the class has many fields.
Is there an easy way to achieve that?

Given an arbitrary type, convert it to JSON using its runtime type only.
Given an arbitrary JSON, convert it to Maps and Lists and primitives.

It would be cool if Moshi would throw an exception if a field was annotated with @NonNull but fromJson() would assign null to it.

This one took us sometime to narrow down. So while stubbing out a custom converter class for a fast setup before starting with TDD, I wrote the implementation as the following:

@FromJson
public MyClass read(JsonReader reader) throws IOException {
    // TODO: write actual implementation
    return new MyClass();
}

The program crashed with an OOM exception and we had no idea why until we tried adding the following:

@FromJson
public MyClass read(JsonReader reader) throws IOException {
    // TODO: write actual implementation
    while(reader.hasNext()) {
        reader.skipValue();
    }
    return new MyClass();
}

Seems like the class that calls this converter via reflection, is waiting for it to consume it all while continuously allocating memory(?). I might be wrong, but I believe this is a bug.

I would love to use the new @Json annotation introduced by #68, but aside from the snapshot releases it is not included in any official release yet. Would you consider publishing a v0.10 release from the current master until all remaining issues for v1.0 are resolved?

I've a scenario where I need to know a peer in order to properly interpret the other. In this case, there's a type field and a value field. The value could be a short, int, long, or double. I don't want to depend on field declaration order.

I was thinking about how to defer the decision since JsonReader smartly handles numbers.

Ex.

JsonReader valueReader = null;
switch (reader.nextName()) {
  case "type":
    type = Type.fromValue(reader.nextString());
    break;
  case "value":
    valueReader = reader.bufferValue(); // similar to peek, except consumes the next value
    break;
--snip--

switch (type) {
  case I64:
    value = valueReader.nextLong();
    break;
  case DOUBLE:
    value = valueReader.nextDouble();
    break;
--snip--

This will be useful when throwing exceptions due to data mismatch problems.

Hello,

I am currently trying to replace Gson in one of my projects with Moshi - everything is working great so far, but there is one point I'd like to ask for a bit of guidance.

I am often using PVectors from http://pcollections.org/ to have immutable Lists in my data objects. For Gson I was using this:

public class PCollectionsAdapterFactory implements TypeAdapterFactory {

@Override
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> typeToken) {

    Type type = typeToken.getType();
    if (typeToken.getRawType() != PVector.class
            || !(type instanceof ParameterizedType)) {
        return null;
    }

    Type elementType = ((ParameterizedType) type).getActualTypeArguments()[0];
    TypeAdapter<?> elementAdapter = gson.getAdapter(TypeToken.get(elementType));
    return (TypeAdapter<T>) createPVectorAdapter(elementAdapter);
}

private <T> TypeAdapter<PVector<T>> createPVectorAdapter(final TypeAdapter<T> elementAdapter) {
    return new TypeAdapter<PVector<T>>() {
        public void write(JsonWriter out, PVector<T> value) throws IOException {
            if (value == null) {
                out.nullValue();
                return;
            }

            out.beginArray();
            for (T element : value) {
                elementAdapter.write(out, element);
            }
            out.endArray();
        }

        public PVector<T> read(JsonReader in) throws IOException {
            if (in.peek() == JsonToken.NULL) {
                in.nextNull();
                return null;
            }

            PVector<T> result = TreePVector.<T>empty();
            in.beginArray();
            while (in.hasNext()) {
                T element = elementAdapter.read(in);
                result = result.plus(element);
            }
            in.endArray();
            return result;
        }
    };
}
}

For Moshi I rewrote this like this:

public class PCollectionsAdapterFactory implements JsonAdapter.Factory {

@Override
public JsonAdapter<?> create(Type type, Set<? extends Annotation> annotations, Moshi moshi) {
    Class<?> rawType = Types.getRawType(type);

    if (!annotations.isEmpty()) return null;
    if (rawType == PVector.class) {
        return newPVectorAdapter(type, moshi);
    } else {
        return null;
    }
}

private static <T> JsonAdapter<PVector<T>> newPVectorAdapter(Type type, Moshi moshi) {
    Type elementType = Types.collectionElementType(type, Collection.class);
    JsonAdapter<T> elementAdapter = moshi.adapter(elementType);
    return new PVectorAdapter<T>(elementAdapter);
}

private static class PVectorAdapter<T> extends JsonAdapter<PVector<T>> {
    private final JsonAdapter<T> elementAdapter;

    public PVectorAdapter(JsonAdapter<T> elementAdapter) {
        this.elementAdapter = elementAdapter;
    }

    @Override
    public PVector<T> fromJson(JsonReader reader) throws IOException {
        if (reader.peek() == JsonReader.Token.NULL) {
            reader.nextNull();
            return null;
        }

        PVector<T> result = TreePVector.<T>empty();
        reader.beginArray();
        while (reader.hasNext()) {
            T element = elementAdapter.fromJson(reader);
            result = result.plus(element);
        }
        reader.endArray();
        return result;
    }

    @Override
    public void toJson(JsonWriter writer, PVector<T> value) throws IOException {
        if (value == null) {
            writer.nullValue();
            return;
        }

        writer.beginArray();
        for (T element : value) {
            elementAdapter.toJson(writer, element);
        }
        writer.endArray();
    }
}
}

And this seems to work fine, too. However I had to copy your whole Types class to my package to have access to Types.getRawType and Types.collectionElementType - do you have any plans to make all these methods public for use in custom JsonAdapter.Factorys or am I just using Moshi wrong?

Also, this seemed to work:

PVector<Channel> channels = startupResponse.channelMetaData.channelList;
try {
    final String channelsAsString = mMoshi.adapter(PVector.class).toJson(channels);
    Timber.d("CHANNELS AS JSON:\n%s", channelsAsString);
    final PVector<Channel> channelsAsPVectorAgain = mMoshi.adapter(PVector.class).fromJson(channelsAsString);
    Timber.d("CHANNELS AS PVECTOR AGAIN:\n%s", channelsAsPVectorAgain);
}

The crucial part here is adapter(PVector.class) - for Gson I always had to do something like new TypeToken<PVector<Channel>>(){}.getType() to deserialize a generic class. Is Moshi just easier that way or am I doing something wrong?

Any replies greatly appreciated!

David

I noticed a bunch while making some other changes.

Classes can declare the following methods:

 <any access modifier> Object toJson();
 <any access modifier> public static Object fromJson(Object json);

If they do, Moshi honors ‘em. The returned objects are then themselves converted to JSON. The idea is that if you’re implementing a library class (say OkHttp’s Request) it can define it’s own JSON form without a compile-time dependency on Moshi. If this works I’ll talk to Tatu (Jackson) and Inder/Joel (gson) to try to make it a universally-supported pattern.

JSON standards defined in [http://www.json.org/](this link) states that an object can be null indeed. As null can be assigned to an String is there a way to do something like jsonReader.nextStringOrNull() ?

I keep getting an

error fetching results: com.squareup.moshi.JsonDataException: Expected BEGIN_OBJECT but was BEGIN_ARRAY at path $

I defined my Moshi adapter like this:
public class SearchJsonAdapter { @FromJson public List<Phrase> fromJson(SearchResult result) { return result.phrases; } }

My Retrofit2 api is defined like this:
@GET ("ac")Observable<Result<SearchResult>> fetchPhrases( @Query("q") String query, @Query("format") String format);

And the model for SearchResult is defined like this:
public final class SearchResult { public List<Phrase> phrases; }

So I am trying to fetch the following JSON: https://ac.duckduckgo.com/ac/?q=android&format=json
which would output this in the browser:

[
-{
phrase: "android device manager"
},
-{
phrase: "android"
},
{
phrase: "android apps"
}
]

Do you know what I did wrong in the Moshi code?

Of course I understand there is no field naming strategy in moshi, but what is the reason?
I mean it seems very critical stuff for ordinary android or java developer because we don't want to create exceptional naming rule.

We have an adapter in place to process byte[] as a base64 / base32 string. Everything works fine when a class has a byte[] field, but parsing json fails on Android when we have a field of type Map<String, byte[]>. The code below works in the standard JVM but fails on Android.

  class Base32Adapter extends JsonAdapter<byte[]>  {
    @Override
    public byte[] fromJson(JsonReader reader) throws IOException {
      String string = reader.nextString();
      return new BigInteger(string, 32).toByteArray();
    }

    @Override
    public void toJson(JsonWriter writer, byte[] bytes) throws IOException {
      String string = new BigInteger(bytes).toString(32);
      writer.value(string);
    }
  }

  static class FavoriteBytes {
    @Json(name = "Bytes")
    public Map<String, byte[]> keys;
  }

  @Test public void customAdapter() throws Exception {
    String jsonString = "{\"Bytes\":{\"jesse\":\"a\",\"jake\":\"1\"}}";
    Moshi binaryMoshi = new Moshi.Builder().add(byte[].class, new Base32Adapter()).build();
    FavoriteBytes fav = binaryMoshi.adapter(FavoriteBytes.class).fromJson(jsonString);

    assertThat(fav.keys).containsOnlyKeys("jesse", "jake");
    assertThat(fav.keys.get("jesse")).isEqualTo(new byte[] { 0xa });
    assertThat(fav.keys.get("jake")).isEqualTo(new byte[] { 0x1 });
  }

On Android, this fails with exception:

E/MoshiTest(18268): com.squareup.moshi.JsonDataException: Expected BEGIN_ARRAY but was STRING at path $.Keys.jesse
E/MoshiTest(18268):     at com.squareup.moshi.JsonReader.beginArray(JsonReader.java:346)
E/MoshiTest(18268):     at com.squareup.moshi.ArrayJsonAdapter.fromJson(ArrayJsonAdapter.java:53)
E/MoshiTest(18268):     at com.squareup.moshi.JsonAdapter$1.fromJson(JsonAdapter.java:68)
E/MoshiTest(18268):     at com.squareup.moshi.MapJsonAdapter.fromJson(MapJsonAdapter.java:68)
E/MoshiTest(18268):     at com.squareup.moshi.MapJsonAdapter.fromJson(MapJsonAdapter.java:29)
E/MoshiTest(18268):     at com.squareup.moshi.JsonAdapter$1.fromJson(JsonAdapter.java:68)
E/MoshiTest(18268):     at com.squareup.moshi.ClassJsonAdapter$FieldBinding.read(ClassJsonAdapter.java:183)
E/MoshiTest(18268):     at com.squareup.moshi.ClassJsonAdapter.fromJson(ClassJsonAdapter.java:144)
E/MoshiTest(18268):     at com.squareup.moshi.JsonAdapter$1.fromJson(JsonAdapter.java:68)
E/MoshiTest(18268):     at com.squareup.moshi.JsonAdapter.fromJson(JsonAdapter.java:33)
E/MoshiTest(18268):     at com.squareup.moshi.JsonAdapter.fromJson(JsonAdapter.java:37)
...

When you try to serialize Map<String,String>, Kotlin actually generates wildcards, i.e. ? extends String. This seems to confuse Moshi
Moshi tries to generate an adapter for ? extends String which is the type of the Map's values but doesn't check the default adapters.
Thanks to @cypressious for his remarks

Hello, I have some jsons which sometimes contain huge values (1+ MB), as images are base64 encoded in them. I know its not the best solution, but its on the server so lets say we cannot change it for now.
For example [{"image": "iAmALongBase64EncodedString"}]
What is the best way to decode the value and store it in a file for later use? Currently I parse that json value as a String, then write it to a file as an InputStream. However some devices get Out of Memory at parsing to a string as it can be huge. Is there any way of skipping the String part and write the value in a file as stream? Thanks!

An example:

class Customer {
  @Json("full name")
  String fullName;
}

(We won't support general policy-based name mangling, like converting camelCase to snake_case or kabob-case)

Specifically how do you get an adapter for this type of structure?

Quasi code for reference:

String json = "[
{ "name" : "foo" },
{ "name" : "bar"}
]";

public class Name {
public final String name;
...
}

Moshi moshi = new Moshi.Builder().build().
JsonAdapter<???> adapter = moshi.adapter(??)
List names = adapter.fromJson(json);

Installed by default; user-replaceable by default.
https://www.ietf.org/rfc/rfc3339.txt

I have raw JSON on disk that I want to embed in side a wrapper JSON. It would be awesome to do something like (explicitly choosing horrible names and hook points before and after)

File fileWithJson = ...;
JsonWriter json = new JsonWriter(sink);
json.beginObject();
json.name("foo").aboutToAddExternalValue();
sink.writeAll(Okio.source(fileWithJson));
json.valueAddedExternallyTrustMeItsFine();
json.closeObject();

Or even:

json.name("foo").value(Okio.source(...))

JsonReader & JsonWriter handles utf8 values (and bytes) internally...which is good for performance reasons

Does it make sense (performance/safety) to have a public api for utf8 ByteString/chars/CharSequence values for JsonReader/Jsonwriter, to prevent object allocation/garbage collection/unnecessary transformations ?

If possible, I would like to avoid this :
{code}
writer.value(charSequence.toString())
{code}

Another difference between Gson and Moshi is, that Gson allows its @SerializedName annotation on enums:

enum CustomEnum {
  @SerializedName("not_a") a,
  b
}

@Test
public void testSerializedName() throws Exception {
  CustomEnum a = gson.getAdapter(CustomEnum.class).fromJson("not_a");
  assertThat(a).isEqualTo(CustomEnum.a);
}

@Test
public void testJson() throws Exception {
  CustomEnum a = moshi.adapter(CustomEnum.class).fromJson("not_a");
  assertThat(a).isEqualTo(CustomEnum.a);
}

This will work with Gson. Unfortunately, Moshi does not process its @Json annotation on enums.

Is this a desired behaviour is this just a missed edge case?

My code:

1. POJO class
   public class LoginResponse {

public Integer ClientID;
public String message;
public String result;
public String sessionID;

}

2. My usage moshi library:

                        Moshi moshi = new Moshi.Builder()
                                .add(new LoginJsonAdapter())
                                .build();
                        Log.v("moshi", "moshi 2.3.2");
                        JsonAdapter<LoginResponse> jsonAdapter = moshi.adapter(LoginResponse.class);
                        LoginResponse loginResponse = jsonAdapter.fromJson(mSringModelResponse);

3. My actual JSon from server, example:

mSringModelResponse is simple java String.

{"ClientID":1,"message":"","result":"OK","sessionID":"0213a0c8-01fff99-4f5c-ad96-e8ffddrreerra0cfac0"}

4. but nothing happen, I'll see it on my LogCat. Should I convert mSringModelResponse in UTF-8 format? Or what is problem?

Please, help.

We have a case where our API has labeled a parameter as "default", which is a protected keyword that can't be used as a variable name in Java. Given that you can't rename parameters with @SerialedName, as in Gson, is there some way to maybe prepend a character like "_" to get around this?

I am having an issue when an api returns an object or an array of objects depending on the number of results.
This behaviour currently crashes moshi with Expected BEGIN_ARRAY but was BEGIN_OBJECT
I am using the yahoo geolocation api and I have no way to alter the way it works.
Currently I use a custom JsonAdapter.Factory

Api example that responds with

• an array Springfield
• a list Blytheville

Check the json field called place in examples above

Currently if someone creates a custom implementation of java.lang.reflect.Type (with a custom JsonAdapter respectively) the utility method Types.getRawType() will throw a IllegalArgumentException doe to support only for Class<?>, ParameterizedType e.t.c. A workaround for this would be to just use type instanceof MyCustomType in the adapter factory, but then the custom adapter has to be first in line, so that other adapters don't throw through Types.getRawType().

This is ok till the point when users of an api that defines the custom type (which is fully private and is an implementation detail) add their own adapters.

By trying to implement the ParameterizedType puts the curtom adapter in conflict with Moshi's collection adapters, and doesn't work more sophisticated use cases like MyCustomType[List<MyCustomType>].