groupon / mongo-deep-mapreduce Goto Github PK

Hi there,

Great library. Really useful.

However, I run into OOM errors when reading data from my raw mongo dump. Some facts:

1. My collection creates 30 extents, and hence, 30 mappers.
2. My whole database is around 200G and this collection is a significant chunk of that data.
3. I am running on a single node hadoop cluster which can run up to 2 tasks in parallel
4. I have set mapred.child.java.opts=-Xmx2G
5. The error occurs when reading records. See [1]

Have you run this library with datasets of that size? Interestingly, if I run the code with LocalJobRunner, it succeeds implying that when the mappers are run serially, there is no problem. It appears that even though we are running 2 tasks at a time, the mappers are created up front and they load in a lot of data which causes the OOM error.

Any pointers?

Adil

[1] Stacktrace:
2013-09-19 20:35:06,031 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError: Java heap space
at com.groupon.mapreduce.mongo.in.Extent.getBuffer(Extent.java:137)
at com.groupon.mapreduce.mongo.in.Record.(Record.java:63)
at com.groupon.mapreduce.mongo.in.Extent.getFirstRecord(Extent.java:124)
at com.groupon.mapreduce.mongo.in.Extent$1.(Extent.java:158)
at com.groupon.mapreduce.mongo.in.Extent.iterator(Extent.java:157)
at com.groupon.mapreduce.mongo.in.MongoRecordReader.initialize(MongoRecordReader.java:59)
at com.groupon.mapreduce.mongo.in.MongoInputFormat.createRecordReader(MongoInputFormat.java:166)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.(MapTask.java:488)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)