bower / decompress-zip Goto Github PK

I want to decompress my archive only if there is a certain file inside. So i call unzipper.list() and check the files in unzipper.on('list')

unzipper.on('list', (files) => { if(fileFound(files){ unzipper.extract({path: folderPath}); }) });

If I do it like that, I got the following error:
"fd must be a file descriptor"

If I just call unzipper.extract() from outside the 'list' event, it works fine.

If I download the zip file from the release at https://github.com/google/material-design-icons/releases/tag/2.0.0, and run that directly through the standard extraction from this readme, I get the following error in Node:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

Any ideas on this one welcome!

Need to go through and rewrite streams code in a backwards compatible manner.

I'm running into an issue with decompress-zip. It works fine on most of the zip files my server generates for packaging files, but on some of the larger files (nearing or exceeding 4.5 GB) it's spitting out an error message:

• Possibly unsupported ZIP platform type, 140
• Possibly unsupported ZIP platform type, 44

When I try to manual unzip the files, they work fine.

graceful-fs versions lower than v4 reevaluate native fs module source, which is not supported.
You should update your graceful-fs dependency to v4 or later.

Tracking issue: nodejs/node#5213.

We use Decompress-Zip module to extract some packages of large size and it fails with EMFILE: too many open files.
Issue Link

Zip files containing directories with identical names but different capitalization will not unzip correctly on windows os. For example

project/Test
project/test

will be unzipped by 7-zip into the same folder on windows. Decompress-zip will store them as two separate paths in the cache and then test both for existance before creating either of them. Neither test will return true and decompress will try and create both, succeeding on the first then mkdir will fail on the second because it already exists. This could be fixed by lowercasing the cached dir name on window systems. Of course having identically named directories (except for the case) unpacked into the same directory has its own problems, but this is how 7-zip appears to work and is probably better than throwing an exception.

I don't think we should expose promises. While promises can be used inside, they should not be used in the API as per node standards.

Also, I think extract should return a stream or a event-emitter, similar to node-tar emitting data resulted from the parse of the zip (entries, etc). The regular end and close events would be fired: end for done, close for done + finish writing to disk.

Linux file name is case sensitive.

I have a program that extracts something when it starts, later you can clear the extracted folder, download a new archive and re-extract it. Sometimes this re-extraction fails, but it never does when the programs starts.

I kept getting this error:

{ [Error: ENOENT, open '/Users/dan/dev/projects/touchlay/daemon/build/Packages/touchlay/demolauncher/css/app.css']
  errno: 34,
  code: 'ENOENT',
  path: '/Users/dan/dev/projects/touchlay/daemon/build/Packages/touchlay/demolauncher/css/app.css' }

As app.css was yet to be created by the unzipper, I figured that ENOENT in this case means that the css/ folder doesn't exist. So I checked and it didn't. Then I added the following filter:

filter: function (file) {
  if (file.type == 'Directory') mkdirp(package_path + '/' + file.path);
  return true;
}

package_path being /Users/dan/dev/projects/touchlay/daemon/build/Packages/touchlay/demolauncher.

What I don't understand is, why does the library sometimes seem to create the folder and sometimes it doesn't. It might have to do with other code I have in my program so if you need more information just ask.

We've run into a situation where an archive created on Unix can sometimes create a zip file where the external file attribute is 0. This causes this line to fail since 0 isn't in the list of types. Much like how archives created in Windows type property defaults to 'File', the Unix decompression should as well.

Like node-tar, it would be handy to be able to extract contents inside a dir directly into your destination.

Some zip files, like http://code.angularjs.org/1.0.7/angular-1.0.7.zip contain : and , in filenames which are not allowed in Windows.

We should have an option, that somehow allows to change the final names of the entries based on a custom function that would be called for each entry.

Hi,
what would be the best way to filter out the dreaded '__MACOSX' folder?
I tried filter: ( file ) => file.filename !== "__MACOSX"but that didn't work.
Thanks, Dirk

Detecting whether a file is a directory or not should be based on the attribute information. At the moment I just check if the path ends with a /.

Can we change the coding style to match bower's?

I have decompress-zip running on a server, and I can imagine a scenario in which a zipfile is created with a symlink to ../../../etc/passwd, for example. I have not actually verified this attack yet, but I looked at the symlink support and it appears that this would be possible.

Hello i have a issue.
When i try extrac a file zip with contain a folder with name "Abc (SOMBody)" that will not create folder

When i specify a strip option, I get the following error:

[TypeError: Cannot read property 'path' of undefined]

The code that generates the error is below:

var DecompressZip = require('decompress-zip');

var sourcePath = './My-Archive.zip';
var targetPath = './temp';
var unzipper = new DecompressZip(sourcePath);

unzipper.on('error', function (err) {
    console.log('Caught an error');
    console.log(err);
});

unzipper.on('progress', function (fileIndex, fileCount) {
    console.log('Extracted file ' + (fileIndex + 1) + ' of ' + fileCount);
});

unzipper.extract({
  path: targetPath,
  filter: function (file) {
    return file.type !== 'SymbolicLink';
  },
  strip: 1
});

The My-Archive.zip file I am using can be downloaded from here:
https://s3.amazonaws.com/jimtars/My-Archive.zip

The zip file was created using 7zip on Windows 8.1.

E.g.

node -e "var DecompressZip = require('decompress-zip'), unzipper = new DecompressZip('blah.zip'); unzipper.extract({ path: './blah' });"

My executable files in ./blah are rw-r--r--, should be rwxr-xr-x

npm ERR! Failed to parse json
npm ERR! Unexpected end of input
npm ERR! File: /Users/Ilan/.npm/decompress-zip/0.0.4/package/package.json
npm ERR! Failed to parse package.json data.
npm ERR! package.json must be actual JSON, not just JavaScript.
npm ERR!
npm ERR! This is not a bug in npm.
npm ERR! Tell the package author to fix their package.json file. JSON.parse

for a zip file with 273 files, it emit such an error.
is there any way to deal with?

on the side, is there a function get zip entry as java zip lib?

thanks for reply

Hi,

I'm trying to extract a .zip file with 50mb and I got this error. Could anyone help me?

EMFILE: too many open files, open '/Users/sergiovilar/hh/plugin-desktop/build/releases/App/osx32/App.app/Contents/Resources/app/node_modules/jquery/src/sizzle/test/data/empty.js'

Steps to reproduce (just an example):

• Download http://cdn.sencha.com/ext/gpl/ext-4.2.1-gpl.zip (~68m) and save it on C:.
• cd C:\ && npm install [email protected]
• Create C:\decompress.js with content as follows:

var DecompressZip = require('decompress-zip');
var unzipper = new DecompressZip("C:\\ext-4.2.1-gpl.zip")
unzipper.on('error', function (err) {
    console.log('Caught an error', err);
});
unzipper.on('extract', function (log) {
    console.log('Finished extracting');
});
unzipper.extract({
    path: 'C:\\',
    follow: false,
    filter: function (file) {
        return file.type !== "SymbolicLink";
    }
});

• Run C:\>node decompress.js:

Caught an error { [Error: EMFILE, open 'C:\ext-4.2.1.883\resources\
ext-theme-gray\images\grid\grid3-hd-btn-left.gif']
  errno: 20,
  code: 'EMFILE',
  path: 'C:\\ext-4.2.1.883\\resources\\ext-theme-gray\\images\\
grid\\grid-hd-btn-left.gif' }

Note that [email protected] also wasn't up to the task (which means neither 1.2.7 nor 1.3.2 versions of Bower are able to install dependencies like that (though tar.gz decompression works fine)).

ENV: OS: Windows XP [Version 5.1.2600], node: v0.10.26

Is there any plans in the pipeline to add a streaming interface to this module?

Have a 'follow' option (name not final) that causes symlinks stored in the zip file to be translated into files/folders.

After decompressed the files inside the zip file has new timestamp as current decompressing time. The original created time for each file is lost.

Currently I'm using decompress-zip version 0.1.0 and 0.3.0, but can't find a LICENSE file in the repo of ver 0.1.0 and 0.3.0.
Could you add the LICENSE file in the repo of ver 0.1.0 and 0.3.0?
Or is it possible to use the LICENSE file of the latest version as the license for ver 0.1.0 and 0.3.0?
Thanks in advance.

To be able to filter symlinks.

See: https://github.com/bower/bower/blob/master/lib/util/extract.js#L40

follow would also be interesting to implement.

There should be options to restore the original permissions/ownership/etc. of files when they are extracted.

Just want to ask about closing a zip file after decompression, what was the reason to not close the opened file?
Right now the file cannot be deleted, moved or renamed after decompression until the process exit.
Is it on purpose or a bug?

Vulnerable versions of decompress-zip are affected by the Zip-Slip vulnerability, an arbitrary file write vulnerability. The vulnerability occurs because decompress-zip does not verify that extracted files do not resolve to targets outside of the extraction root directory.

I get an error EBUSY event triggered when my unzipping has finished. The extract event is called, but I always get the EBUSY fatal error. What might be causing that?

Seems like max file entry length of 4096 is hard coded. For large zip files we get errors like:

File entry unexpectedly large: 36366 (max: 4096)

I'm using

unzipper.on('error', function (err) {
  console.log('Caught an error', err);
});
...

And after that all error events are caught by this listener so i have to manually remove it.

giving this error. What is possibly wrong ?

On MacOS there is an issue with symlinks after extract.
Paths are absolute and in case that some files are moved they become invalid.

Example is with packed Electron.app.
After extracting all symlinks are absolute and if Electron.app is moved to another folder it becomes corrupted.

Solution would be to provide option to select between relative and absolute symlink paths.
Node.js fs already has great utility for that path.relative.

I am using this package to extract zip file to a specific folder in both Mac OS and Windows.

In Windows, it's quite fine and I can wait for extracting the zip file.
But the problem is it's too slow and long to extract the zip file in MacOS.

Zip file size: around 1GB
Total file counts in the zip: 3000 files

Please let me know how can I improve speed?

I'm attempting to unzip the darwin mac app for atom shell .

I've tried a ton of npm modules as well as issued some bugs but no response.

• adm-zip cthackers/adm-zip#118
• node-unzip EvanOxfeld/node-unzip#68
• decompress-unzip
• buckle
• yauzl

Every time I try to unzip the atom shell app with node / programmatically I get this error.

I opened up mac Console and can see it logs the following when I try to run:

1/21/15 4:02:11.440 PM com.apple.xpc.launchd[1]: (com.github.atom-shell.255064[19701]) Could not find and/or execute program specified by service: 13: Permission denied: /Users/thomas/Desktop/wallpaper/wallpaper-app/build/Atom.app/Contents/MacOS/Atom
1/21/15 4:02:11.440 PM com.apple.xpc.launchd[1]: (com.github.atom-shell.255064[19701]) Service setup event to handle failure and will not launch until it fires.
1/21/15 4:02:11.440 PM Finder[339]: spawn_via_launchd() failed, errno=111 label=com.github.atom-shell.255064 path=/Users/thomas/Desktop/wallpaper/wallpaper-app/build/Atom.app/Contents/MacOS/Atom flags=1 : LaunchApplicationClient.cp #1136 LaunchApplicationViaLaunchDJobLabel() q=com.apple.root.default-qos
1/21/15 4:02:11.440 PM Finder[339]: spawn_via_launchd() failed, errno=111 label=com.github.atom-shell.255064 path=/Users/thomas/Desktop/wallpaper/wallpaper-app/build/Atom.app/Contents/MacOS/Atom flags=1

I was really happy to see that this library had support for symlinks, and it seems to look as it they are working working. However the app still won't open. Thoughts?

If a archive contains a lot of dirs this leads to the situation where the overall progress cannot be correctly interpreted. Can be bad for GUIs and logfiles...

Decompress zip is slow. I mean, like, really slow.

❯ time bower install extjs=http://cdn.sencha.com/ext/commercial/ext-4.2.1-commercial.zip
bower not-cached    http://cdn.sencha.com/ext/commercial/ext-4.2.1-commercial.zip#*
bower resolve       http://cdn.sencha.com/ext/commercial/ext-4.2.1-commercial.zip#*
bower download      http://cdn.sencha.com/ext/commercial/ext-4.2.1-commercial.zip
bower extract       extjs#* ext-4.2.1-commercial.zip
bower resolved      http://cdn.sencha.com/ext/commercial/ext-4.2.1-commercial.zip#e-tag:5e5ef2426

real    46m37.575s
user    46m12.850s
sys 2m13.090s

For reference:

❯ time unzip ext-4.2.1-commercial.zip

real    0m4.470s
user    0m2.190s
sys 0m1.000s

Hello everyone,

I am desperately looking for a solution to extract a large zip file (3Gb), and trying some APIs to find the best. Importing decompress-zip is OK, but when I write "var DecompressZip = require('decompress-zip');" I have the message "Import error, can't find file:
./zlib_bindings".

I am coding on Stackblitz with angular 7. I had the same problem with yauzl, unzip-stream, extract-zip, AdmZip...

Thank you in advance !

What if I don't want to monkey patch fs? I have no choice if this module depends on graceful-fs. Worse, if another module depends on decompress-zip, and I depend on that module, I now am (possibly unknowingly) monkey patching fs.

How to solve large files unzip error log Array buffer allocation failed

it would be sweet to use this client-side, but some things would need to change first:

• remove assumption that a filename will be passed in and a folder will be created + extracted to
• instead have an abstract interface that requests byte ranges (e.g. what getBuffer is doing) but is decoupled from fs. that way it can be hooked up to FileReader in the browser. require('fs') would have to move to separate node-only file from the extraction logic. the extraction logic would be generic so that it can run in the browser or in node
• also abstract the extraction so that instead of passing in a destination folder you can just get a readable stream for each file file extraction

I have tested using the following:

import DecompressZip from 'decompress-zip'
import extract from 'extract-zip'
import path from 'path'

extract('ngrok.zip', { dir: path.resolve('~/ezip')}).then(_ => console.log('DONE'))

var unzipper = new DecompressZip('ngrok.zip')

unzipper.on('error', function (err) {
    console.log('Caught an error');
});

unzipper.on('extract', function (log) {
    console.log('Finished extracting');
});

unzipper.on('progress', function (fileIndex, fileCount) {
    console.log('Extracted file ' + (fileIndex + 1) + ' of ' + fileCount);
});

unzipper.extract();

The ngrok in ./ezip/ngrok works but the one extracted using decompress-zip fails results in:

./ngrok
[1]    52121 killed     ./ngrok

I'm on MacOS 11.6 (Big Sur)

Is tehre any method to close() or destroy() a job when in progress?

Could you fill the git tags and optionally fill changelog for them?