michel-kraemer / bson4jackson Goto Github PK

View Code? Open in Web Editor NEW

288.0 24.0 86.0 985 KB

🦬 A pluggable BSON generator and parser for the Jackson JSON processor.

Home Page: https://michelkraemer.com/binary-json-with-bson4jackson

License: Apache License 2.0

Java 100.00%

jackson bson java jackson-json-processor mongodb binary json

bson4jackson's Introduction

BSON for Jackson

This library adds support for BSON to the Jackson JSON processor.

BSON is a binary representation of JSON. It is well known as the main exchange and persistence format of MongoDB.

Quick start

Just create a Jackson ObjectMapper with a BsonFactory as follows:

ObjectMapper mapper = new ObjectMapper(new BsonFactory());

For more information, you may read my bson4jackson tutorial or the complete documentation of Jackson.

Download

bson4jackson binaries are available from the GitHub releases page.

You may also use Maven to download bson4jackson:

<dependencies>
    <dependency>
        <groupId>de.undercouch</groupId>
        <artifactId>bson4jackson</artifactId>
        <version>2.15.1</version>
    </dependency>
</dependencies>

If you are using Gradle, you may add the following snippet to your build.gradle:

dependencies {
    implementation 'de.undercouch:bson4jackson:2.15.1'
}

Compatibility

The latest version of bson4jackson is backward compatible to all versions of Jackson 2.x released up to date. It should be compatible to newer versions as well. If you experience compatibility issues, just let me know.

If you are looking for a version compatible to Jackson 1.x, please use bson4jackson 1.3.0. It's the last version for the 1.x branch. bson4jackson 1.3.0 is compatible to Jackson 1.7 up to 1.9.

Compiling

Execute the following command to compile the library and to run the unit tests:

./gradlew test

The script automatically downloads the correct Gradle version so you won't have to do anything else. If everything runs successfully, you may create a .jar library:

./gradlew jar

The library will be located under build/libs.

License

bson4jackson is licensed under the Apache License, Version 2.0.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

bson4jackson's People

Contributors

Stargazers

Watchers

bson4jackson's Issues

BsonParser.nextToken() throws JsonParseException instead of returning null

The javadoc for JsonParser states that the parser should return null when there are no more tokens available, but this isn't what happens:

public class BsonTest {
    public static void main(String[] args) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        BsonFactory bsonFactory = new BsonFactory();
        BsonGenerator generator = bsonFactory.createJsonGenerator(out);
        generator.writeStartObject();
        generator.writeStringField("myField", "myValue");
        generator.writeEndObject();
        generator.close();

        JsonParser parser = bsonFactory.createJsonParser(out.toByteArray());
        while (parser.nextToken() != null) {
            System.out.println(parser.getCurrentToken());
        }
    }
}

Gives:

START_OBJECT
FIELD_NAME
VALUE_STRING
END_OBJECT
Exception in thread "main" org.codehaus.jackson.JsonParseException: Found element outside the document
 at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@ce102dc; pos: 23]
    at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:125)
    at BsonTest.main(BsonTest.java:20)

Tested with baaec01

Performance problems when deserializing

I have run some performance tests and compared bson4jackson with jackson when deserializing using the stream api.

Surprisingly bson4jackson is much slower for me.
I tried to deserialize a very simple file 500.000 times.

Plain Jackson needs 330-390ms while bson4jackson takes between 870 and 1010ms.

Since your benchmark results (http://www.michel-kraemer.com/binary-json-with-bson4jackson) seem to be much better, there must be something wrong on my side. Here is the code I am using:

The file I am parsing (has been generated using bson4jackson

00000000026964000a00000043616e6f6e205261770008646570656e64656e74000003657874656e73696f6e000000000002657874656e73696f6e0004000000637232000864656661756c7400010264656c696d6974657200020000002e000000

And here the code used to deserialize:

      JsonParser parser = factory.createParser( contentSample );

      assertEquals( JsonToken.START_OBJECT, parser.nextToken() );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "id", parser.getCurrentName() );
      assertEquals( JsonToken.VALUE_STRING, parser.nextToken() );
      String id = parser.getText();
      assertEquals( "Canon Raw", id );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "dependent", parser.getCurrentName() );
      assertEquals( JsonToken.VALUE_FALSE, parser.nextToken() );
      boolean dependent = parser.getBooleanValue();
      assertFalse( dependent );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "extension", parser.getCurrentName() );
      assertEquals( JsonToken.START_OBJECT, parser.nextToken() );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "extension", parser.getCurrentName() );
      assertEquals( JsonToken.VALUE_STRING, parser.nextToken() );
      String extension = parser.getText();
      assertEquals( "cr2", extension );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "default", parser.getCurrentName() );
      assertEquals( JsonToken.VALUE_TRUE, parser.nextToken() );
      boolean isDefault = parser.getBooleanValue();
      assertTrue( isDefault );

      assertEquals( JsonToken.FIELD_NAME, parser.nextToken() );
      assertEquals( "delimiter", parser.getCurrentName() );
      assertEquals( JsonToken.VALUE_STRING, parser.nextToken() );
      String delimiter = parser.getText();
      assertEquals( ".", delimiter );

      assertEquals( JsonToken.END_OBJECT, parser.nextToken() );
      assertEquals( JsonToken.END_OBJECT, parser.nextToken() );
      assertNull( parser.nextToken() );

      parser.close();

      FileType type = new FileType( id, new Extension( delimiter, extension, isDefault ), dependent );
      assertNotNull( type );

The factory is only created once:

                        BsonFactory jsonFactory = new BsonFactory();
                        jsonFactory.enable( BsonGenerator.Feature.ENABLE_STREAMING );

Any ideas?

Creating a Mongo query using a String

Hi,

Mongo driver provides a JSON.parse(String query) method that returns a DBObject.
With it, one can do the following:

public void find() {
    DBObject query = JSON.parse("{value:{$exists:true}}");
    DBCursor cursor = collection.find(query);
}

Using bson4jackson it is easy to un/marshall objects into DBObject:

public void save() {
    DBObject document = marshall(new Friend("John", 24));
    collection.save(document);
}

ObjectMapper mapper = new ObjectMapper(new BsonFactory());

public DBObject marshall(Object obj) throws Exception {
    ByteArrayOutputStream output = new ByteArrayOutputStream();
    mapper.writer().writeValue(output, obj);
    return new LazyWriteableDBObject(output.toByteArray(), new LazyBSONCallback());
}

But because queries are String, I don't find how to do the same with them. So, how to marshall a query into a DBObject like JSON.parse() with Jackson and bson4jackson? This will help a lot.

Thanks so much.

Bonus question: is there a way to marshall as a String? Like:

String document = marshall(new Friend("John", 24));
Assert.assertEquals("{name: 'John', age: 24}", document);

Error when deserialising BSON containing an array as the root element

Details of the error on stackoverflow: http://stackoverflow.com/questions/17320921/deserialise-array-of-bson-objects-as-root-object-in-java

In short I get a Can not deserialize instance of x out of START_OBJECT token when using an array as the root element of a document. The equivalent JSON object would look like this:

[{"id":....},{"id":....},{"id":....}....]

Can this be changed to work on arrays as root elements?

gradle build doesn't properly package sources

When building in gradle 1.2 I get a bson4jackson-sources.sources artifact instead of a bson4jackson-sources.jar artifact. This looks like a bug in the artifacts section of the build file.

This fixes it for me:

 artifacts {
     archives jar
-    archives(packageJavadoc) {
-        type = 'javadoc'
-    }
-    archives(packageSources) {
-        type = 'sources'
-    }
+    archives packageJavadoc
+    archives packageSources
 }

Jackson 2 support

Hi Michel,

This issue is really just a placeholder for supporting Jackson 2. Jackson 2 is a major API breaking upgrade (the package names have all changed), and I'd like to start testing with it in the Mongo Jackson Mapper so my first task was to upgrade bson4jackson to it. I've done this in a branch, if you want you can pull that branch across:

https://github.com/jroper/bson4jackson/tree/jackson-2

Jackson 2 is still in early stages, no releases in maven yet and will still be making more breaking changes, so there's no rush here, I just wanted to create this issue in case anyone was looking for Jackson 2 support so they could get it from my branch.

Cheers,

James

MalformedInputException on valid BSON.

Hitting a MalformedInputException on a particular bson file. Unfortunately, I can't share it since it has customer data in it. The BSON seems to be valid (bsondump has no problem with it). I'll try to dig in deeper when I get a chance, but logging an issue in case it's useful.

Exception in thread "main" java.lang.RuntimeException: Input length = 1
at com.fasterxml.jackson.databind.MappingIterator.next(MappingIterator.java:124)
at scala.collection.JavaConversions$JIteratorWrapper.next(JavaConversions.scala:575)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)
at bson2json.App$.main(App.scala:37)
at bson2json.App.main(App.scala)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:277)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:293)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:227)
at de.undercouch.bson4jackson.BsonParser.readString(BsonParser.java:570)
at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:292)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:422)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
at com.fasterxml.jackson.databind.MappingIterator.nextValue(MappingIterator.java:189)
at com.fasterxml.jackson.databind.MappingIterator.next(MappingIterator.java:120)
... 5 more

java.lang.NoClassDefFoundError: org/bson/types/MinKey

When I try to connect to MongoDB, I get a Java exception. I'm curious if Jongo, mongo-java-driver, or bson4jackson is the locus of the problem.

System specs and error trace at:

bguerout/jongo#161

BsonGenerator & binary subtypes

It looks like the BsonParser handles the binary subtypes but the BsonGenerator doesn't seem to have any support for that, specifically for being able to write UUIDs out as binary types. Is this something that's planned, and if not, how hard would it be to add it?

The constructor ObjectMapper(BsonFactory) is undefined

Hello,

The following lines:

ObjectMapper mapper = new ObjectMapper(new BsonFactory());
mapper.writeValue(baos, bob);

from the following sample
result in an error

The constructor ObjectMapper(BsonFactory) is undefined

Why this project needs a custom ObjectId object?

I cannot understand why this plugins needs to use a own ObjectId object and I also don't understand why it haven't a factory method to instantiate the ObjectiId. Actually I create new ObjectId instances using the org.bson.types.ObjectId and not seems very much convenient.

Problem to add Binary Streaming

you have plenty of methods which can be extended, for example:

JsonToken handleBinary()

but I can't get variable "_in" to get what I would, also class BsonParser.Context isn't available and so on ... can u make it protected that developers can do necessary changes? I want to implement own streaming of big files for BSON, it's really necessary

Date and 'polymorphic' class causes JsonGenerationException: BsonSerializer can only be used with BsonGenerator

I got an issue when updating to Jongo 1.3.0 that seems to be related to #67, I've created a question here: www.stackoverflow.com/questions/42004332/date-and-polymorphic-class-causes-jsongenerationexception-bsonserializer-can

Once again it seems that the order of the properties causes the issue and I only see the problem with java.util.Date. bson4jackson/2.8.0-SNAPSHOT gave a different problem, also described in the SO post.

EOF exception reading serialized object

In https://github.com/bimargulies/bson-bug you will find what seems to me to be a simple test case that writes one object's worth of data to a bson file, and then tries to read it back, but fails with an EOFException.

Introducing record boundaries

I am creating a giant quantity that I wish to treat as atomic records. My plan is to write each record's worth into a BDB recno record.

Do I need to use a new bson4jackson generator for each record, or can I capture the output byes for each record?

ObjectID parsing should be big endian

According to the ObjectID spec, the time and counter fields of ObjectID should be stored big endian:

"Note that the timestamp and counter fields must be stored big endian unlike the rest of BSON."

from http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-BSONObjectIDSpecification

bson4jackson is decoding/encoding them using little endian. Additionally, machine/pid are not integers, they are arbitrary ids, so are actually byte arrays. Decoding them as little endian doesn't make sense. The MongoDB org.bson ObjectID treats the ObjectID as 3 big endian integers, I think this is a better way of treating things.

BigDecimal number lose precision due to conversion to floating point

Michel,
Is it possible to add a 'Feature' to serialize/deserialize BigDecimal types as strings (or some other non lossy format) rather than floats? We're finding our data changes after going through bson4jackson.
Thanks
Nigel

BSONParser reads past end of document

The BSONParser assumes that it is entitled to read to the end of the stream. While this is usually a valid assumption for reading JSON, BSON is designed to be used as part of chatty binary protocols, for example, it's used by MongoDB, where a single stream will have more than just a single BSON document going across it. That's why the first 4 bytes of the stream is the size of the document, so that appropriate buffering can be done while reading, without interferring with the rest of the stream.

The BSONParser ignores the size of the document, and reads past the end of the document (as part of its buffering), which results in it consuming things it should not consume. I've solved this by wrapping the input stream that goes into it with an input stream that reads the first 4 bytes, decodes them into the size, outputs them as the first 4 bytes, and returns EOF once that size has been read, but really I think the BSONParser should handle this itself.

java.lang.ArrayIndexOutOfBoundsException at de.undercouch.bson4jackson.io.DynamicOutputBuffer

Hello, I'm encountering out of bounds runtime exception during BSON generation

Caused by: java.lang.ArrayIndexOutOfBoundsException: -32768
    at java.util.ArrayList.get(ArrayList.java:324)
    at de.undercouch.bson4jackson.io.DynamicOutputBuffer.getBuffer(DynamicOutputBuffer.java:262)
    at de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(DynamicOutputBuffer.java:550)
    at de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(DynamicOutputBuffer.java:531)
    at de.undercouch.bson4jackson.BsonGenerator._writeFieldName(BsonGenerator.java:347)
    at de.undercouch.bson4jackson.BsonGenerator._writeArrayFieldNameIfNeeded(BsonGenerator.java:328)
    at de.undercouch.bson4jackson.BsonGenerator.writeNull(BsonGenerator.java:543)
    at com.nuodb.migrator.resultset.format.bson.BsonOutputFormat.writeValues(BsonOutputFormat.java:100)
    at com.nuodb.migrator.resultset.format.FormatOutputBase.writeRow(FormatOutputBase.java:136)

Source file without license header

Hi
The following source files are without license header.

./src/main/java/de/undercouch/bson4jackson/serializers/BsonSerializers.java

Please, confirm the licensing of code and/or content/s, and add license headers.

https://fedoraproject.org/wiki/Packaging:LicensingGuidelines?rd=Packaging/LicensingGuidelines#License_Clarification

Thanks in advance
Regards

Support BSON Decimal128 type

MongoDB 3.4 added a new BSON type: Decimal128

Java support came in version 3.4 of the driver, and was not backported to 2.x. References:

Ability to write binary values in native binary

It seems that since Jackson was primarily written for JSON, it has to convert binary values to Base 64 to be able to write them to disk properly.

However, this is not necessary with BSON, as when BSON values are written, they are prefixed with a length field describing their length. Therefore, 20-30% disk space savings can be achieved by not converting binary values to Base 64 and using the length header properly.

Can this be implemented and achieved in bson4jackson?

Trouble with Strings larger than 64k

It appears that by reusing the CharsetDecoder requires a reset at least when trying to handle strings that will require multiple calls to handle.

Caused by: java.lang.IllegalStateException: Current state = CODING_END, new state = CODING
at java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:951)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:537)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:311)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:264)
at de.undercouch.bson4jackson.BsonParser.readString(BsonParser.java:506)
at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:228)

Polymorphism issue?

Using 2.6.0, serialize to BSON (and send to Mongodb) with an object mapper that has mixins that use @Class for polymorphism. When I go to read back (currently by parsing from a string from the object from the cursor), the @Class is missing. Is this surprising?

Missing codec setup in createJsonGenerator API

I claim that the first of these should do the same thing as the second. The corresponding JsonFactory methods do so.

public BsonGenerator createJsonGenerator(OutputStream out) throws IOException {
    return new BsonGenerator(_generatorFeatures, _bsonGeneratorFeatures, out);
}

@Override
public BsonParser createJsonParser(InputStream in) throws IOException {
    BsonParser p = new BsonParser(_parserFeatures, in);
    ObjectCodec codec = getCodec();
    if (codec != null) {
        p.setCodec(codec);
    }
    return p;
}

Can't serialize Long value

Please help me with number de/serialization

    ObjectMapper objectMapper = new ObjectMapper(new BsonFactory());
    objectMapper.registerModule(new BsonModule());

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    objectMapper.writeValue(baos, 1L);

    ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
    Long v = objectMapper.readValue(bais, Long.class); // exception is here!

I would like to use bson4jackson and be able to de/serialize number and other simple types. Is it possible?

MalformedInputException on object read using Java 1.8

When runing the following code a MalformedInputException occurs when the stored object is deserialized (stacktrace below). The exception only occurs in a Java 1.8 environment (e.g., 1.8.0_25).

 public class MappedObject {
      @JsonProperty()
       public String text;
       ...  
 }

 MappedObject mappedObject = new MappedObject();
 mappedObject.text = new String(new byte[] {-16,-97,-110,-103}, "UTF-8");

 com.fasterxml.jackson.databind.ObjectMapper mapper = org.mongojack.internal.MongoJackModule.configure(new ObjectMapper());
 mapper.writeValue(..., mappedObject);

 org.mongojack.JacksonDBCollection<MappedObject, String> wrappedCollection = ...
 //exception
 mappedObject = wrappedCollection.find(<some query to read mappedObject>);

Caused by: java.nio.charset.MalformedInputException: Input length = 3
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:293)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:227)
at de.undercouch.bson4jackson.BsonParser.readString(BsonParser.java:570)
at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:292)
at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:172)
at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:154)
at com.fasterxml.jackson.databind.deser.std.StringCollectionDeserializer.deserialize(StringCollectionDeserializer.java:19)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:525)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:106)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:242)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2965)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1626)
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:77)
at com.mongodb.Response.(Response.java:85)
at com.mongodb.DBPort$1.execute(DBPort.java:141)
at com.mongodb.DBPort$1.execute(DBPort.java:135)
at com.mongodb.DBPort.doOperation(DBPort.java:164)
at com.mongodb.DBPort.call(DBPort.java:135)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:292)
... 38 more

Error when trying to deserialize BSON into a EXISTING_PROPERTY polymorphic class

I ran into an problem when trying to deserialize BSON into a polymorphic class that uses JsonTypeInfo.As.EXISTING_PROPERTY.

Groovy script:

@Grab('de.undercouch:bson4jackson:2.7.0')

import com.fasterxml.jackson.annotation.JsonSubTypes
import com.fasterxml.jackson.annotation.JsonTypeInfo
import com.fasterxml.jackson.databind.ObjectMapper
import de.undercouch.bson4jackson.BsonFactory
import de.undercouch.bson4jackson.BsonModule

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXISTING_PROPERTY, property = 'type')
@JsonSubTypes([
    @JsonSubTypes.Type(name = 'a', value = A),
    @JsonSubTypes.Type(name = 'b', value = B)
])
abstract class Base {
    UUID id
    String type
}

class A extends Base {}

class B extends Base {}


def mapper = new ObjectMapper(new BsonFactory()).registerModule(new BsonModule())
def bytes = mapper.writeValueAsBytes(new A(id: UUID.randomUUID()))
mapper.readValue(bytes, Base)

Throws the stacktrace:

Caught: com.fasterxml.jackson.core.JsonGenerationException: BsonSerializer can only be used with BsonGenerator
com.fasterxml.jackson.core.JsonGenerationException: BsonSerializer can only be used with BsonGenerator
        at de.undercouch.bson4jackson.serializers.BsonSerializer.serialize(BsonSerializer.java:37)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2436)
        at com.fasterxml.jackson.databind.util.TokenBuffer.writeObject(TokenBuffer.java:849)
        at com.fasterxml.jackson.databind.util.TokenBuffer.copyCurrentEvent(TokenBuffer.java:1006)
        at com.fasterxml.jackson.databind.util.TokenBuffer.copyCurrentStructure(TokenBuffer.java:1048)
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:97)
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:142)
        at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:42)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3788)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2872)
        at com.fasterxml.jackson.databind.ObjectMapper$readValue$1.call(Unknown Source)
        at so.run(so.groovy:27)

Hints:

When I use JsonTypeInfo.As.PROPERTY the issue goes away.
When I don't register BsonModule the issue goes away.
This was occurring on JDK 7 but not JDK 8. After trying to debug a bit, this appears to be because of reflection field order.

EOFException when reading last document in BSON mongodump file using MappingIterator

The following code will fail on a BSON dump file (produced with mongodump) after the last BSON object is read.

ObjectMapper mapper = new ObjectMapper(new BsonFactory());
MappingIterator<BSONObject> iterator = (MappingIterator) mapper.reader(BasicBSONObject.class).readValues(bsonFile);

while(iterator.hasNext()) {
   BSONObject object = iterator.next();
}

Stacktrace:

java.lang.RuntimeException
    at com.fasterxml.jackson.databind.MappingIterator.hasNext(MappingIterator.java:112)
    at ...
Caused by: java.io.EOFException
    at de.undercouch.bson4jackson.io.LittleEndianInputStream.readFully(LittleEndianInputStream.java:85)
    at de.undercouch.bson4jackson.io.LittleEndianInputStream.readInt(LittleEndianInputStream.java:144)
    at de.undercouch.bson4jackson.BsonParser.handleNewDocument(BsonParser.java:355)
    at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:169)
    at com.fasterxml.jackson.databind.MappingIterator.hasNextValue(MappingIterator.java:158)
    at com.fasterxml.jackson.databind.MappingIterator.hasNext(MappingIterator.java:108)
    ... 29 more

I think this is because it is trying to read the length when no more records exist

ObjectId mismatch

There is a difference in de.undercouch.bson4jackson.types.ObjectId and org.bson.types.ObjectId (Version 3.4.2). org.bson.types.ObjectId holds an additional processIdentifier.

gradle 2.x

Hi
can you provide a settings(.gradle) file with the following content:
rootProject.name="bson4jackson"
the reasons of this request are explained here
https://discuss.gradle.org/t/rootproject-name-in-settings-gradle-vs-projectname-in-build-gradle/5704/2
I used for fix: echo 'rootProject.name="bson4jackson"' >> settings.gradle

There is another problem related to javadoc generation. with Java 8 build fails.

:javadoc
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:116: warning: no @return
    public final BsonFactory configure(BsonGenerator.Feature f, boolean state) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:128: warning: no @return
    public BsonFactory enable(BsonGenerator.Feature f) {
                       ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:138: warning: no @return
    public BsonFactory disable(BsonGenerator.Feature f) {
                       ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:146: warning: no @param for f
    public final boolean isEnabled(BsonGenerator.Feature f) {
                         ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:156: warning: no @return
        public final BsonFactory configure(BsonParser.Feature f, boolean state) {
                                 ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:168: warning: no @return
        public BsonFactory enable(BsonParser.Feature f) {
                           ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:178: warning: no @return
        public BsonFactory disable(BsonParser.Feature f) {
                           ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:186: warning: no @param for f
        public final boolean isEnabled(BsonParser.Feature f) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:199: warning: no @param for out
        protected BsonGenerator _createJsonGenerator(Writer out, IOContext ctxt) {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:199: warning: no @param for ctxt
        protected BsonGenerator _createJsonGenerator(Writer out, IOContext ctxt) {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:199: warning: no @return
        protected BsonGenerator _createJsonGenerator(Writer out, IOContext ctxt) {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:207: warning: no @param for data
        protected BsonParser _createJsonParser(byte[] data, int offset, int len, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:207: warning: no @param for offset
        protected BsonParser _createJsonParser(byte[] data, int offset, int len, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:207: warning: no @param for len
        protected BsonParser _createJsonParser(byte[] data, int offset, int len, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:207: warning: no @param for ctxt
        protected BsonParser _createJsonParser(byte[] data, int offset, int len, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:207: warning: no @return
        protected BsonParser _createJsonParser(byte[] data, int offset, int len, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:215: warning: no @param for in
        protected BsonParser _createJsonParser(InputStream in, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:215: warning: no @param for ctxt
        protected BsonParser _createJsonParser(InputStream in, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:215: warning: no @return
        protected BsonParser _createJsonParser(InputStream in, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:223: warning: no @param for r
        protected BsonParser _createJsonParser(Reader r, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:223: warning: no @param for ctxt
        protected BsonParser _createJsonParser(Reader r, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:223: warning: no @return
        protected BsonParser _createJsonParser(Reader r, IOContext ctxt) {
                             ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:256: warning: no @param for out
        protected BsonGenerator _createUTF8JsonGenerator(OutputStream out, IOContext ctxt) throws IOException {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:256: warning: no @param for ctxt
        protected BsonGenerator _createUTF8JsonGenerator(OutputStream out, IOContext ctxt) throws IOException {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:256: warning: no @return
        protected BsonGenerator _createUTF8JsonGenerator(OutputStream out, IOContext ctxt) throws IOException {
                         ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:256: warning: no @throws for java.io.IOException
        protected BsonGenerator _createUTF8JsonGenerator(OutputStream out, IOContext ctxt) throws IOException {
                                ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:87: warning: no @param for src
    protected BsonFactory(BsonFactory src, ObjectCodec codec) {
              ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:87: warning: no @param for codec
    protected BsonFactory(BsonFactory src, ObjectCodec codec) {
              ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/BsonFactory.java:73: warning: no @param for oc
    public BsonFactory(ObjectCodec oc) {
           ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/CountingInputStream.java:40: warning: no @param for in
        public CountingInputStream(InputStream in) {
               ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/DynamicOutputBuffer.java:62: error: unexpected end tag: </p>
 * </p>
   ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/LittleEndianInputStream.java:63: warning: no @param for in
        public LittleEndianInputStream(InputStream in) {
               ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/StaticBuffers.java:118: warning: no @param for key
        public ByteBuffer byteBuffer(Key key, int minSize) {
                          ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/StaticBuffers.java:118: warning: no @param for minSize
        public ByteBuffer byteBuffer(Key key, int minSize) {
                          ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/StaticBuffers.java:118: warning: no @return
        public ByteBuffer byteBuffer(Key key, int minSize) {
                          ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/StaticBuffers.java:134: warning: no @param for key
        public void releaseByteBuffer(Key key, ByteBuffer buf) {
                    ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/io/StaticBuffers.java:134: warning: no @param for buf
        public void releaseByteBuffer(Key key, ByteBuffer buf) {
                    ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/deserializers/BsonDeserializer.java:50: warning: no @throws for java.io.IOException
    public abstract T deserialize(BsonParser bp, DeserializationContext ctxt)
                      ^
~/rpmbuild/BUILD/bson4jackson-2.5.0/src/main/java/de/undercouch/bson4jackson/deserializers/BsonDeserializer.java:50: warning: no @throws for com.fasterxml.jackson.core.JsonProcessingException
    public abstract T deserialize(BsonParser bp, DeserializationContext ctxt)
                      ^
1 error
38 warnings
:javadoc FAILED

Can you add in the build.gradle this workaround?

javadoc {
    options.encoding = 'UTF-8'
    options.addStringOption('Xdoclint:none', '-quiet')
}

Regards
Thanks in advance

BsonParser#getText throws ClassCastException if token is not a string

This is a pretty straightforward bug in 2.0.0.

    @Override
    public String getText() throws IOException, JsonParseException {
        Context ctx = _contexts.peek();
        if (ctx == null || ctx.state == State.FIELDNAME) {
            return null;
        }
        if (ctx.state == State.VALUE) {
            return ctx.fieldName;
        }
        return (String)ctx.value;
    }

The last line should be calling ctx.value.toString() instead of trying to cast to a String (possibly w/ a null check first).

Infinite loop in de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(int, String)

Hello,
I'm randomly encountering an infinite loop in de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(int, String)

I've got a thread that never ends and takes 100% of one thread of the CPU (on the same set of data, the problem doesn't always occur).

Here is the call stack of the thread :

application-akka.actor.default-dispatcher-12 [RUNNABLE]
de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(int, String)
de.undercouch.bson4jackson.io.DynamicOutputBuffer.putUTF8(String)
de.undercouch.bson4jackson.BsonGenerator._writeCString(String) 
de.undercouch.bson4jackson.BsonGenerator._writeString(String)
de.undercouch.bson4jackson.BsonGenerator.writeString(String)
com.fasterxml.jackson.databind.ObjectWriter.writeValue(OutputStream, Object)
org.jongo.marshall.jackson.JacksonEngine.marshall(Object)
org.jongo.Insert.marshallDocument(Object)
org.jongo.Insert.createDBObjectToInsert(Object)
org.jongo.Insert.save(Object)
org.jongo.MongoCollection.save(Object)
models.audits.PageEntityCollection.save(PageEntity)

I have noticed that the thread repetitively call to the CharsetEncoder encode() method.
So I think the thread is block in
while (in.remaining() > 0) {
...
}

Any ideas where the issue comes from, or how can I debug it ?

bson4jackson should be able to handle a unnested value

An error occurred in the following case:

ObjectMapper objectMapper = new ObjectMapper(new BsonFactory());
Integer i = 100;
byte[] bytes = objectMapper.writeValueAsBytes(i);
objectMapper.readValue(bytes, Object.class); // an error occurred!

The error is as follows:

Exception in thread "main" java.io.EOFException
    at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUnsignedByte(LittleEndianInputStream.java:114)
    at de.undercouch.bson4jackson.io.LittleEndianInputStream.readByte(LittleEndianInputStream.java:107)
    at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:261)
    at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:490)
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702)
    at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2807)
    at Main.main(Main.java:15)

Support for CharacterEscapes

I was wondering if there are any plans to support the CharacterEscapes feature introduced in Jackson 1.8?

I'm working on trying to escape the '.'s and '$'s in field names so they can be properly stored in mongo. I took a good long look at the source and although I can set the correct escapes on a JsonGenerator, they will have no effect in the BsonGenerator implementation.

Thanks, let me know what you think!

Unable to deserialize binary values properly

I've written a simple example project called bson-example-java which attempts to demonstrate how to simply serialize and deserialize object trees in Java using Jackson with bson4jackson.

Unfortunately, it seems that there is a bug in bson4jackson's implementation of reading back binary values.

Serialization

I'm serializing org.bson.ObjectId objects using the following serializer:

@Override public void serialize (ObjectId value, JsonGenerator jgen, 
        SerializerProvider provider) throws IOException, JsonGenerationException {
    byte[] bytes = value.toByteArray();

    logger.debug("Writing {]-byte binary ObjectId: {}, hex: {}", bytes.length, bytes,
            encodeHexString(bytes));

    jgen.writeBinary(bytes);
}

When I run my example, I can see from the output that it's doing things properly:

11:21:00.993 [main] DEBUG BinaryObjectIdSerializer - Writing 12-byte binary ObjectId: [83, -110, 6, -116, -28, -80, 103, 125, 59, -2, -110, -50], hex: 5392068ce4b0677d3bfe92ce
11:21:01.009 [main] DEBUG BinaryObjectIdSerializer - Writing 12-byte binary ObjectId: [83, -110, 6, -116, -28, -80, 103, 125, 59, -2, -110, -52], hex: 5392068ce4b0677d3bfe92cc
11:21:01.009 [main] DEBUG BinaryObjectIdSerializer - Writing 12-byte binary ObjectId: [83, -110, 6, -116, -28, -80, 103, 125, 59, -2, -110, -51], hex: 5392068ce4b0677d3bfe92cd

I can also use bsondump to see my generated BSON tree and everything looks right:

$ bsondump /tmp/example.bson
{ 
    "_id" : { "$binary" : "U5IGjOSwZ307/pLO", "$type" : "00" }, 
    "title" : "The Beginning of Everything", 
    "date" : Date( 0 ), 
    "tags" : [ 
        { "name" : "beginning", "_id" : { "$binary" : "U5IGjOSwZ307/pLM", "$type" : "00" } }, 
        { "name" : "first", "_id" : { "$binary" : "U5IGjOSwZ307/pLN", "$type" : "00" } } 
    ], 
    "body" : "Wow, such much is wow yes!" 
}

Un base64-ing the binary id values and converting to hexadecimal yields:

5392068ce4b0577d3bfe92ce
5392068ce4b0677d3bfe92cc
5392068ce4b0677d3bfe92cd

Therefore, I've determined that the serialization is working properly and that it's only the deserialization that we encounter a problem.

Deserialization

The bug is experienced when attempting to read back the binary data into memory during deserialization. Instead of receiving back 12 bytes, I get either 10 or 11 bytes (it seems that depending on the value of the bytes, I get 10 or 11 bytes... weird). Here is my deserializer:

@Override public ObjectId deserialize (JsonParser jp, DeserializationContext ctxt) 
        throws IOException, JsonProcessingException {
    byte[] value = jp.getBinaryValue();

    logger.debug("Read {}-byte ObjectId: {}; hex: {}", value.length, value,
        encodeHexString(value));

    return new ObjectId(value);
}

Here's the output from my logs:

11:49:43.397 [main] DEBUG BinaryObjectIdDeserializer - Read 11-byte ObjectId: [91, 66, 64, 49, 52, 53, 52, 54, 50, 98, 99]; hex: 5b42403134353436326263
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: need 12 bytes (through reference chain: org.tkassembled.example.bson.data.Article["_id"])
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:232)
    at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:197)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1420)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:244)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118)
    at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1270)
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:865)
    at org.tkassembled.example.bson.BSONReaderApplication.main(BSONReaderApplication.java:42)
Caused by: java.lang.IllegalArgumentException: need 12 bytes
    at org.bson.types.ObjectId.<init>(ObjectId.java:203)
    at org.tkassembled.example.bson.deserialize.BinaryObjectIdDeserializer.deserialize(BinaryObjectIdDeserializer.java:35)
    at org.tkassembled.example.bson.deserialize.BinaryObjectIdDeserializer.deserialize(BinaryObjectIdDeserializer.java:1)
    at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:525)
    at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:106)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:242)
    ... 4 more

As is noted, it returns an entirely different ObjectId, and only 11 bytes of data. If I instead change my deserializer to do byte[] value = (byte[]) jp.getEmbeddedObject();, I actually do get the ObjectId properly:

11:52:06.714 [main] DEBUG BinaryObjectIdDeserializer - Read 12-byte ObjectId: [83, -110, 6, -116, -28, -80, 103, 125, 59, -2, -110, -50]; hex: 5392068ce4b0677d3bfe92ce
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.Date out of VALUE_EMBEDDED_OBJECT token
 at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@5a4bb836; pos: 71] (through reference chain: org.tkassembled.example.bson.data.Article["date"])
    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164)
    at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:691)
    at com.fasterxml.jackson.databind.deser.std.StdDeserializer._parseDate(StdDeserializer.java:626)
    at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateBasedDeserializer._parseDate(DateDeserializers.java:166)
    at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:260)
    at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:241)
    at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:525)
    at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:242)
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118)
    at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1270)
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:865)
    at org.tkassembled.example.bson.BSONReaderApplication.main(BSONReaderApplication.java:42)

However, it seems to cause problems with reading the next field, and breaks the deserialization of the rest of the tree.

Therefore, it seems clear that there's a bug in the readBinaryValue logic present in bson4jackson.

Remove workaround for buffer overflow on Travis CI

See travis-ci/travis-ci#5227

Revert commit dfbdc5e

Write byte array

Hello,

How do I write a byte[] with BSON?
I want the binary JSON to contain a byte array that isn't a base64 string.
I know that I can do it with BSON in C#, but how can I do it with bson4jackson?

byte[] myArray = new byte[10];
...
JsonGenerator jg = bsonMapper.getJsonFactory().createJsonGenerator(os);
jg.writeStartObject();
jg.writeStringField("Test", "Yep");
jg.writeByteArrayField("Array", myArray);
jg.writeEndObject();
jg.close();
os.close();

Thanks!

2.7.0 release is missing on Maven Central/JCenter

Unfortunately 2.7.0 is not available in my builds. I checked Maven Central and JCenter, and the package is missing:

https://bintray.com/bintray/jcenter/de.undercouch%3Abson4jackson/view
http://mvnrepository.com/artifact/de.undercouch/bson4jackson

Also the releases page for version 2.7.0 is linking to the 2.6.0 jar

BSON unmarshalling creates a lot of garbage

Hi,

actually i figure out that the used underlying BSON unmarshalling creates a lot of garabage (round 540 MB) to unmashalling 4.7 MB of mongoDB objects. It seems that these StaticByteBuffers are unreachable for the JVM escape analysis.

What can we do pls?

Greetings Steffen

Date should seriazlie as \x09

Hello, I found Date not seriazlied as \x09, but \x12.
My Class is
public class TestClass {
private Date v;
public getV() { return this.test; }
public setV(Date value) { this.v = value; }
}
When I set v to 1970-01-01, the serialize result is
\x10\x00\x00\x00\x12\x76\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.
But it should be
\x10\x00\x00\x00\x09\x76\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00.
I dont known why this will happen, how can I fix this?

Reopen issue #35

Need to reopen #35
I left a comment in there. Since it's marked as closed I'm opening a new one.

Field name lost when using JsonParser.readValueAsTree()

When reading am embedded object/document with JsonParser.readValueAsTree() the _nextToken is set to null which forces a new document in BsonParser.nextToken(). This in turn loses the field name of a following field. Try the test below. It will result in expecting a field_name but getting a start_object.

To fix, only start a new document in nextToken() if there is nether a _currentToken nor _contexts.peek(). See suggested patch at end of test code.

    import org.codehaus.jackson.JsonToken;
    import org.codehaus.jackson.JsonNode;
    @Test
    public void parseEmbeddedDocumentAsTree() throws Exception {
        BSONObject o2 = new BasicBSONObject();
        o2.put("Int64", 10L);

        BSONObject o3 = new BasicBSONObject();
        o3.put("Int64", 11L);

        BSONObject o1 = new BasicBSONObject();
        o1.put("Obj2", o2);
        o1.put("Obj3", o3);

        BSONEncoder enc = new BSONEncoder();
        byte[] b = enc.encode(o1);

        ByteArrayInputStream bais = new ByteArrayInputStream(b);
        BsonFactory fac = new BsonFactory();
        ObjectMapper mapper = new ObjectMapper(fac);
        fac.setCodec(mapper);

        BsonParser dec = fac.createJsonParser(bais);

        assertEquals(JsonToken.START_OBJECT, dec.nextToken());

        assertEquals(JsonToken.FIELD_NAME, dec.nextToken());
        assertEquals("Obj2", dec.getCurrentName());
        assertEquals(JsonToken.START_OBJECT, dec.nextToken());
        JsonNode obj2 = dec.readValueAsTree(); 
        assertEquals(1, obj2.size());
        assertNotNull(obj2.get("Int64"));
        assertEquals(10L, obj2.get("Int64").getValueAsLong());

        assertEquals(JsonToken.FIELD_NAME, dec.nextToken());
        assertEquals("Obj3", dec.getCurrentName());
        assertEquals(JsonToken.START_OBJECT, dec.nextToken());

        assertEquals(JsonToken.FIELD_NAME, dec.nextToken());
        assertEquals("Int64", dec.getCurrentName());
        assertEquals(JsonToken.VALUE_NUMBER_INT, dec.nextToken());
        assertEquals(11L, dec.getLongValue());

        assertEquals(JsonToken.END_OBJECT, dec.nextToken());

        assertEquals(JsonToken.END_OBJECT, dec.nextToken());
    }

    public JsonToken nextToken() throws IOException, JsonParseException {
        Context ctx = _contexts.peek();
        if (_currToken == null && ctx == null) {
           _currToken = handleNewDocument(false);
        } else {
            _tokenPos = _counter.getPosition();
            if (ctx == null) {
                throw new JsonParseException("Found element outside the document", getTokenLocation());
            }
...etc

Empty output when using in Spring's AbstractJackson2HttpMessageConverter

I try using bson4jackson in Spring MVC web application to convert domain objects at a REST endpoint.

I define bson convertor as follows:

public class Jackson2BsonHttpMessageConverter extends AbstractJackson2HttpMessageConverter {
    public Jackson2BsonHttpMessageConverter() {
        super(new ObjectMapper(new BsonFactory()), new MediaType("application", "bson"));
    }
}

Spring's AbstractJackson2HttpMessageConverter then converts a domain object in the following way:

protected void writeInternal(Object object, Type type, HttpOutputMessage outputMessage) {
    JsonGenerator generator = this.objectMapper.getFactory().createGenerator(outputMessage.getBody(), encoding);
...
    ObjectWriter objectWriter = this.objectMapper.writer();
...
    objectWriter.writeValue(generator, value);
...
}

And that results in empty output.

I recreated this behavior in a unit-test:

@Test
public class Bson4JacksonTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(Bson4JacksonTest.class);

    @Test
    public void bsonSerializationTest() throws IOException {
        TestPerson testObject = new TestPerson(12L, "Joe", "+79201112233", "84952334455");
        ObjectMapper om = new ObjectMapper(new BsonFactory());
        byte[] result1 = om.writeValueAsBytes(testObject); // returns 89 bytes, looks ok
        ByteArrayOutputStream os = new ByteArrayOutputStream();
        JsonGenerator g = om.getFactory().createGenerator(os);
        om.writeValue(g, testObject);
        g.flush();
        byte[] result2 = os.toByteArray(); // returns 0 bytes, looks wrong
        LOGGER.debug("Bson representation: {}", new String(result2));
        assertTrue(result2.length > 0); // fails
        assertEquals(result2, result1); // fails
    }

    public static class TestPerson {
        private Long id;
        private String name;
        private Set<String> phoneNumbers;
        // constructors and accessors are ommited
    }
}

Unfortunately, I cannot debug it deep enough to find a root cause.

I think that it is a bug because the same approach of using Spring's AbstractJackson2HttpMessageConverter works with jackson-dataformat-msgpack library that integrates to jackson in the same way.

Could you please fix that behavior or suggest a workaround?

I have tried bson4jackson 2.6.0 and 2.7.0 with jackson 2.6.3, 2.6.4, 2.6.5, 2.7.0 and 2.7.3

Create 2.8 version for Jackson 2.8

I know that bson4jackson is quite compatible across multiple Jackson versions, but it would be nice to have a newer version that specifies Jackson 2.8.x dependencies, as users try to match minor versions (for good reason too, since not all Jackson components can retain cross-minor-version deps).
Doing this also allows bson module to implement latest hooks for low-level parser/generator.

Unable to read a BSON document with a MinKey inside

Since 2.5.0 a BSON document with a MinKey can no longer be read from a StaticBufferedInputStream wrapped by a LittleEndianInputStream.

See pull request #50

Threading problems with CharsetDecoder

I've encountered a another issue with the CharsetDecoder, I wrote a small test in my branch to reproduce the problem. When more than one thread is trying to decode UTF strings, the static decoder which is keeping state will throw the following error.

Caused by: java.lang.IllegalStateException: Current state = CODING_END, new state = CODING
at java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:951)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:537)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:312)
at de.undercouch.bson4jackson.io.LittleEndianInputStream.readUTF(LittleEndianInputStream.java:264)
at de.undercouch.bson4jackson.BsonParser.readString(BsonParser.java:506)
at de.undercouch.bson4jackson.BsonParser.nextToken(BsonParser.java:228)

I'm not sure what the penalty of instantiating a new decoder per instance is, so I made a change to make it ThreadLocal. If this solution makes sense to you, I can generate another pull request.