Large Import of MultiSheets with Chunk and Queued Reading the ReadChunk Crashes about laravel-excel HOT 8 CLOSED

If it crashes on the part that PhpSpreadsheet tries to load the file, there's not much we can do/fix in this package. Not sure if it's the file size, or something else with the file is wrong, as it fails on the first time reading the file.

from laravel-excel.

Hello @patrickbrouwers

Thanks for your quick reply.
Can you suggest me how can i debug or solve it.

Or if you need any more information.
I am sure it is some memory issue as

When i remove the withChunkReading and shouldQueue from the Import file

It throws an exception
Symfony\Component\Process\Exception\ProcessSignaledException
The process has been signaled with signal "9".

Any idea if i can handle it with some configuration in excel.config

I tried to do
batch and illumninate

Instead of memory

But it didn't work

from laravel-excel.

Hello @patrickbrouwers

I digged Deeper and found that OS is killing the Process due to high memory usage.
Is there nothing we can do regarding memory

May be because I am using MultiSheet

it is not using ChunkReading.

Can you suggest something for Memory

I found this in kern.log file of OS
kernel: [ 9163.607949] Out of memory: Killed process 94495 (php) total-vm:2974980kB, anon-rss:2835984kB, file-rss:3584kB, shmem-rss:0kB, UID:0 pgtables:5716kB oom_score_adj:0

from laravel-excel.

Perhaps you placed the chunk reading interface on the wrong class. It should be on the sheet

from laravel-excel.

Hello @patrickbrouwers

That is not the case, When I remove the WithChunkReading, ShouldQueue from the Main Class

The ReadChunk is not initialized and a Single Job Does all the Work.
This Case also fails the queue:work with the main Job

Also the ReadChunk Job Class is not in the PHPspreadsheet package it is in our package maatwebsite/excel

So it is something, you guys can help with

Also how is the code working in smaller files.
If that is the case of wrong class

from laravel-excel.

Hello @patrickbrouwers

The is one more update from my side.
I did an experiment and it worked, but the below cannot be the actual solution.

The above sheets i was talking about were all Multisheets
I copied the content of the Main Sheet in the Workbook into a Single Sheet Workbook and then manipulated the Single Sheet Importer.

Now the ReadChunk donot crash due to memory usage.

May be the readchunk is not proper when we use with multisheets, this is a thing where you guys should debug
This happens in multiSheet only when sheet is too heavy,
If you want I can share you the sheet.

This is the new response of the queue:work with Single Sheet (Still has 25,000 rows)

from laravel-excel.

Can you share the file with me?

from laravel-excel.

This bug report has been automatically closed because it has not had recent activity. If this is still an active bug, please comment to reopen. Thank you for your contributions.

from laravel-excel.