Slow When reading more than 65000 records with 10 columns. about excel-streaming-reader HOT 8 CLOSED

There are a lot of factors that can contribute to this, so I don't have enough information from your issue. Can you provide an example spreadsheet, the full set of JVM options you used, and a description of what your application does?

from excel-streaming-reader.

I need an example spreadsheet, not your code. I can't reproduce this behavior without the data you are reading in.

from excel-streaming-reader.

from excel-streaming-reader.

You need to attach the spreadsheet to the issue. Here is a guide: https://help.github.com/articles/file-attachments-on-issues-and-pull-requests/. It looks like you're using email to reply to me and I don't think adding it as an email attachment works.

from excel-streaming-reader.

test2.xlsx

Here is the sample spreadsheet. first read with one sheet of 65000 record.
then copy first sheet and create new sheet paste there. and try to read two
sheet of 130000 records. and see the difference of timing.

from excel-streaming-reader.

I too not able read large excel files with 1 million rows, I am getting java.lang.OutOfMemoryError: Java heap space

I am not able attach my excel file which is around 34MB( getting errror Yowza that's a big file. Try again with a file smaller than 10MB)

from excel-streaming-reader.

@kakadikrishna Moved discussion to #73

from excel-streaming-reader.

@MRabindra10 It consistently takes my computer around 5-5.5 seconds to read 64800 rows. Subsequent reads are slightly faster, averaging about 3.5 seconds (most likely due to the lack of classloading required). Given the volume of data to be processed, I seriously doubt it will ever be able to completely read in under 1 second. As you can see, 1.3 seconds is spent just by the SAX parser to read elements out of the stream in the run on your data (the XMLEventReaderImpl.nextEvent() call):

That particular call can't really be affected by any of the library code, so I'm left with the conclusion that your measurement of 594ms is most likely in error. I'm not entirely sure what's going on with your environment, but are you sure that the code to read from the sheet even executes completely? If you drop out of the loop early, you might see an issue like this.

from excel-streaming-reader.