Fail to connect to mongo from unit test about nosql-unit HOT 12 CLOSED

Hi Joel, yes this is the right place,
it is strange because there is nothing wrong at first look, Moreover in demo project there is an example that is doing exactly the same of you and it works (https://github.com/lordofthejars/nosql-unit/blob/master/nosqlunit-demo/src/test/java/com/lordofthejars/nosqlunit/demo/mongodb/WhenANewBookIsCreated.java) and I am using Ubuntu too.

So let's try some things. First of all if you are using nosqlunit 0.7.3 Mongo Java Driver used is the 2.10.1 don't know if this could affect on your MongoDB version. I have tried with 2.2.0. So let's start a basic test to know what's happening:

Use the previous test but do not use MongoDbRule, only ManagedMongoDb, and then let's try to connect from Mongo Java Driver directly, something like:

@ClassRule
   public static ManagedMongoDb managedMongoDb = newManagedMongoDbRule().mongodPath("/home/joe/prog/mongo/mongodb").build();


   @Test
   public void should_return_NameCollection_with_given_name() {
       MongoClient mongo = new MongoClient("127.0.0.1", 27017);
       mongo.getDB("admin").getStats();

    }

Probably won't work but at least we will know that there is nothing wrong in Mongo Rule. If it works then there is something wring with rule, if not let's try another thing. Let's write insertions:

@ClassRule
   public static ManagedMongoDb managedMongoDb = newManagedMongoDbRule().mongodPath("/home/joe/prog/mongo/mongodb").build();


   @Test
   public void should_return_NameCollection_with_given_name() {
       MongoClient mongo = new MongoClient("127.0.0.1", 27017);
       mongo.getDB("test").getCollection("mycol").insert(....);

    }

If it works then we know that there is something wrong when trying to retrieve stats of Mongo server, if not the problem is still in Mongo instance. And we can only try next thing:

Remove the ManagedMongoDb rule, so no NoSQLUnit classes are present. Then starts you manually MongoDB server by typing exactly the same command line:

/home/joe/prog/mongo/mongodb/bin/mongod --dbpath mongo-dbpath --port 27017 --logpath logpath

Be aware that probably dbpath should be changed to some directory with write privileges. And then re run previous tests but of course without @ClassRule so your test will use the instance you have already started up manually. Then if they work we should see what's happening when NoSQLUnit tries to start MongoDB, if still happens the error, then I will suggest that there is something in MongoDB installation/configuration that does not fit with Mongo Java Driver.

Hope to help you and of course send me your results to try to help you.

Thank you very much for using NoSQLUnit.

Alex.

from nosql-unit.

Hi Alex,

Thanks for the quick answer, really appreciated ! :)

I think I may have an explanation for the error ...

First, I'm using 0.7.3 version with mongo java driver 2.10.1, I have tried running the test from Idea and Eclipse with the same results.

Following your advice, using only ManagedMongoDb and trying to connect from Mongo Java Driver directly gave the same error- note that mongod is started with the command I mentioned earlier.

One interesting point : if I rerun the test after waiting a little bit, the test succeeds and then stop mongod (with java.io.EOFException as mentionned in your tutorial), to sum up (assuming no mongod is running) :

• launching the test two times without waiting leads to 2 test errors.
• launching the test once, waiting some time, relaunching the test, then first test fails but second one succeeds.

I think that the error comes from mongod not being ready to serve requests even though it is started.

The mongod process status when launched is "uninterruptible", running the test in this status leads to an error, whereas if I wait mongod status to be "sleeping" the second test succeeds (process status can be displayed in System Monitor).

When launching mongod manually, it takes almost one minute on my machine before being ready to serve request, see the logpath content below :

Mon Jan 21 21:01:10 [initandlisten] MongoDB starting : pid=16381 port=27017 dbpath=mongo-dbpath 64-bit host=joe-desktop
Mon Jan 21 21:01:10 [initandlisten] db version v2.2.2, pdfile version 4.5
Mon Jan 21 21:01:10 [initandlisten] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Mon Jan 21 21:01:10 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Mon Jan 21 21:01:10 [initandlisten] options: { dbpath: "mongo-dbpath", logpath: "logpath", port: 27017 }
Mon Jan 21 21:01:10 [initandlisten] journal dir=mongo-dbpath/journal
Mon Jan 21 21:01:10 [initandlisten] recover : no journal files present, no recovery needed
Mon Jan 21 21:01:13 [initandlisten] preallocateIsFaster=true 30.86
Mon Jan 21 21:01:15 [initandlisten] preallocateIsFaster=true 35.48
Mon Jan 21 21:01:19 [initandlisten] preallocateIsFaster=true 29.64
Mon Jan 21 21:01:19 [initandlisten] preallocateIsFaster check took 8.601 secs
Mon Jan 21 21:01:19 [initandlisten] preallocating a journal file mongo-dbpath/journal/prealloc.0
Mon Jan 21 21:01:22 [initandlisten]         692060160/1073741824    64%
Mon Jan 21 21:01:25 [initandlisten]         975175680/1073741824    90%
Mon Jan 21 21:01:33 [initandlisten] preallocating a journal file mongo-dbpath/journal/prealloc.1
Mon Jan 21 21:01:36 [initandlisten]         671088640/1073741824    62%
Mon Jan 21 21:01:39 [initandlisten]         912261120/1073741824    84%
Mon Jan 21 21:01:50 [initandlisten] preallocating a journal file mongo-dbpath/journal/prealloc.2
Mon Jan 21 21:01:53 [initandlisten]         713031680/1073741824    66%
Mon Jan 21 21:01:56 [initandlisten]         975175680/1073741824    90%
Mon Jan 21 21:02:03 [websvr] admin web console waiting for connections on port 28017
Mon Jan 21 21:02:03 [initandlisten] waiting for connections on port 27017

If I run the test after "waiting for connections on port 27017" log, it succeeds.

Finally, when ManagedMongoDb and MongoDbRule are commented and mongod is started manually, I was able to get admin stats, insert one document in NameCollection and retrieve it afterwards.
stats{ "serverUsed" : "/127.0.0.1:27017" , "db" : "admin" , "collections" : 0 , "objects" : 0 , "avgObjSize" : 0.0 , "dataSize" : 0 , "storageSize" : 0 , "numExtents" : 0 , "indexes" : 0 , "indexSize" : 0 , "fileSize" : 0 , "nsSizeMB" : 0 , "ok" : 1.0}

The test succeeds since mongod was ready to serve request.

I think NoSQLUnit tries to connect to mongod too early, is there a way to :

• tell NoSQLUnit to wait for mongod to be ready to serve requests ?
• speed up mongod start (without journal maybe )?

Thanks for your help !

from nosql-unit.

With the code below that disables the journal, tests are running successfully :)

@ClassRule
public static ManagedMongoDb managedMongoDb = newManagedMongoDbRule()
                   .mongodPath("/home/joe/prog/mongo/mongodb")
                   .appendSingleCommandLineArguments("--nojournal")
                   .build();

I don't know if it's possible but IMHO the best way to solve the initial problem is for NoSQLUnit to wait for mongod to be ready to serve requests, WDYT ?

from nosql-unit.

Well in fact it is already doing that. The problem is that Stats command is not still ready (this command is used to know if mongod is already started up) so it fails with an exception. I will fix this error by managing exception too.

from nosql-unit.

Seems a nice solution !

What about disabling journal by default in ManagedMongoDb ? (or mentioning in the doc that it takes longer to start with journaling), I'm not sure journaling matters a lot in unit testing where you clean the database.

from nosql-unit.

Hi Joeal well the problem is that depending on MongoDB version and if it is 32 or 64 journaling is enabled by default or not. Anyway I will implement the exception management and I will put no journaling argument line in managed way.

Thanks!!!

from nosql-unit.

I have added nojournal option as default argument. Moreover I have extended the timeout time to 2 minutes. Although this actions, managed MongoDB should be used only in integration tests where journaling should not to be activated, if it is activated and MongoDB takes more than two minutes to startup, a timeout exception will be thrown.

from nosql-unit.

Excellent !
Thanks.

from nosql-unit.

you are welcome next version will contain this fixed problem, if you need quickly I can deliver a version, but I would like to wait until elasticsearch support has been implemented.

Again thanks for helping improving NoSQLUnit and for any problem or question don't hesitate to ask me.

from nosql-unit.

I can wait, no problem.

Thanks for this useful library.

from nosql-unit.

Resolved via upgrade of drivers:

<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongo-java-driver</artifactId>
    <!--  <version>2.7.2</version>  -->
    <version>2.10.1</version>
</dependency>

from nosql-unit.

It took me a day to figure out how to get it works with Mongo-Java-Driver Version 3.0.3 in IntelliJ! Put this to your ManagedMongoDB:

appendSingleCommandLineArguments("--fork")

before calling build().

from nosql-unit.