FATAL ERROR: JS Allocation failed - process out of memory about tar-stream HOT 12 CLOSED

The api seems to indicate that this was using tar-fs and not tar-stream right?

from tar-stream.

sorry, yea, wrong repo

its worth noting that this equivalent code doesnt run out of memory and completes, but takes longer than I think tar-stream would take if it wasnt leaking

var tar = require('tar')
var fstream = require('fstream')
var reader = fstream.Reader({type: "Directory", path: '/Users/max/Downloads/audiobooks/'})
var pack = tar.Pack()
reader.pipe(pack).pipe(tar.Extract({path: '/Users/max/Downloads/audiobooks2'}))

from tar-stream.

Which versions of tar-stream and tar-fs are you using? Also which version of node?

from tar-stream.

[email protected], [email protected], [email protected]

from tar-stream.

Using the latests versions I just copied a 3.7 gb directory using tar-fs in 7.5 seconds on my Macbook with no apparent leak. Using node-tar and the snippet you linked it still hasn't finished (been running for +5min now).
I'll try to see if I can reproduce the issue...

from tar-stream.

Do you have any symlinks in the folder you are trying to copy?

from tar-stream.

Hmmm, no symlinks. I first tried copying a 10gb folder full of mp3s, then a
sub folder of about 1gb of mp3s. Maybe it is related to the quantity of
files?

On Sunday, December 22, 2013, Mathias Buus wrote:

Do you have any symlinks in the folder you are trying to copy?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4#issuecomment-31097152
.

from tar-stream.

Did the folder contain a lot of sub folders?

from tar-stream.

one of the folders that crashes every time: 338 documents, 27 folders, 4,324,597,805 bytes (4.33 GB on disk)

from tar-stream.

I think I've found (and solved) the issue. It was due to a bug being triggered by a long filename (>100 characters) that caused it to explode. It should be fixed now. Could you try upgrading tar-fs to 0.1.2 and try again?

from tar-stream.

nice, copies 485 documents, 25 folders, 9,279,350,569 bytes (9.28 GB on disk) in 38423ms and 338 documents, 27 folders, 4,324,597,805 bytes (4.33 GB on disk) in 17659ms

from tar-stream.

Nice! How long does it take to do it with node-tar?

from tar-stream.