Giter Club home page Giter Club logo

recursive-readdir's Introduction

recursive-readdir

Build Status

Recursively list all files in a directory and its subdirectories. It does not list the directories themselves.

Because it uses fs.readdir, which calls readdir under the hood on OS X and Linux, the order of files inside directories is not guaranteed.

Installation

npm install recursive-readdir

Usage

var recursive = require("recursive-readdir");

recursive("some/path", function (err, files) {
  // `files` is an array of file paths
  console.log(files);
});

It can also take a list of files to ignore.

var recursive = require("recursive-readdir");

// ignore files named "foo.cs" or files that end in ".html".
recursive("some/path", ["foo.cs", "*.html"], function (err, files) {
  console.log(files);
});

You can also pass functions which are called to determine whether or not to ignore a file:

var recursive = require("recursive-readdir");

function ignoreFunc(file, stats) {
  // `file` is the path to the file, and `stats` is an `fs.Stats`
  // object returned from `fs.lstat()`.
  return stats.isDirectory() && path.basename(file) == "test";
}

// Ignore files named "foo.cs" and descendants of directories named test
recursive("some/path", ["foo.cs", ignoreFunc], function (err, files) {
  console.log(files);
});

Promises

You can omit the callback and return a promise instead.

var recursive = require("recursive-readdir");

recursive("some/path").then(
  function(files) {
    console.log("files are", files);
  },
  function(error) {
    console.error("something exploded", error);
  }
);

The ignore strings support Glob syntax via minimatch.

recursive-readdir's People

Contributors

agamemnus avatar ajedi32 avatar akerpelm avatar aymericbeaumet avatar bnb avatar davidcroda avatar davidnguyen11 avatar dpeukert avatar edrex avatar ferhatelmas avatar jergason avatar nikuda avatar panuhorsmalahti avatar rjmunro avatar seeekr avatar thanpolas avatar tobiasbueschel avatar woodyrew avatar wtgtybhertgeghgtwtg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

recursive-readdir's Issues

Ignore files doesnt work

recursive("src/aura", ['.css', '.xml'], function(err, files) { console.log(files) }

When I read files I anywhere see filed css and xml files

Get directories into list

Hi and thanks for this great module!
In order to include directories in the list, i just needed to insert at line 73:
list.push(filePath);

Could we have an option for that or so? fs.readdir also includes folders in the list, i think recursive-readdir should reflect the original behaviour.

Return matching files

Does minimatch allow you to pass something like "!*.txt" to the ignore list?.

I'm am hoping return a list of only matched file types, as if I was doing a recursive "ls *.txt".

Broken symlink returns only `undefined`

When running recursive on a dir with broken symlinks, undefined is returned for each one.

To test, create a dir, test/, and cd into it. Create a dir with touch a. Create a symlink with ln -s a b. Delete a with rm -rf a. Then run recursive on the test/ dir. A single undefined will be returned.

Add v2.2.2 to GitHub releases

I was checking out what I thought was the latest release of recursive-readdir. I even opened a comparison to see what goodies were coming in the next release.

Then I saw this…and realized I had just wasted ≈15 minutes. 🤦🏽‍♂️

Couldja add the latest version to GitHub Releases?

(Note: If you start a commit with Release [v]« semver-version», GitHub will auto-create a GitHub release for you! ☻)

ignore all, authorize callback ?

Hey this tool are awesome thanks.
One thing, we really need support ignore all
Any tips to do this
Example i want allow only if is a files, and the extension is .json and .webm

        const scanDATA2 = () => {
            const recursive = require('recursive-readdir');
            function ignoreFunc(file, stats) {
                const isDirectory = stats.isDirectory();
                if(stats.isDirectory()){
                    return file.contains('SOURCE') || file.contains('source');
                }else
                if(stats.isFile()){
                    return !(['.json','.webm'].contains(file));
                };
              };
              recursive("data2", [ignoreFunc], (err, files) => {
                files.forEach(PATH => {
                    const split = PATH.replace('.','\\').split('\\');
                    const name = split[split.length-2];
                    if(name.contains('-')){ // isMultipack
                        const _name = name.split('-')[0]; // nom sans -0
                        !this.LINKS[_name] && (this.LINKS[_name] = []);
                        this.LINKS[_name].push( {name:name,path:PATH.replace(/\\/g,"/")} );
                    }else{
                        this.LINKS[name] = PATH.replace(/\\/g,"/");
                    };
                });
                return nextStep();
              });
        };

the issue with the ignore files, is if a user add a new file in folder, it will scan.

Make ignores resolve relative to the path passed to recursive-readdir

If I pass an absolute path to recursive-readdir

readdir = require('recursive-readdir')
readdir("/home/ajedi32/some_dir", function(e, l){ console.log(l); })

and I want to ignore a specific file in a subdirectory of that path, like /home/ajedi32/some_dir/folder/file.txt, then I should be able to specify the ignore as "folder/file.txt". The full path of "/home/ajedi32/some_dir/folder/file.txt" shouldn't be required.

vulnerability with minimatch @ 3.0.4, fixed in @ 3.0.5+

Running a container scan report revealed the following high severity vulnerability:

minimatch @ 3.0.4 - Status: fixed in 3.0.5
minimatch package versions before 3.0.5 are vulnerable to Regular Expression Denial of Service (ReDoS). It's possible to cause a denial of service when calling function braceExpand (The regex /\{.*\}/ is vulnerable and can be exploited).

The fix would likely be to update minimatch to v3.0.5+.

Add doNotRecurseIntoDirectory option?

It'd save some time on very big subdirectory trees if you could avoid reading subdirectories that you're going to ignore anyways e.g. node_modules. Better to just avoid reading those directories at all.

Would you be ok with a PR adding this?

Error: ENOENT: no such file or directory, stat '/home/oradicke/.kde/cache-subtux'

Hi jeragason,

nice packet! But I think there is an issue with broken symbolic link.

Error: ENOENT: no such file or directory, stat '/home/oradicke/.kde/cache-subtux'

[oradicke@altona aguano]$ file /home/oradicke/.kde/cache-subtux 
/home/oradicke/.kde/cache-subtux: broken symbolic link to /var/tmp/kdecache-or

What can I do for ignoring broken symbolic links?

Best regards

Olaf

Published version is different

Looks like published version package.json is different from one in the repository. That's why

  1. Package contains test and .travis.yml
  2. "engines": { "node": ">=0.10.0" },

`sort` option

Add an option to sort files and folder

This achieves the same effect as readdr(...).sort(...) but is a lot faster for large directories

Promise support

I tried it. Works great. Would love if this library returned Promises.

Passing a function lists out directories not files on Windows 10

The below code appears to work correctly on OSX, however on Windows it seems to fail.

What I have noticed is that the ignoreFiles function is passed a list of top level subdirectories rather than a list of the files contained within them

getFiles(folder)
 .then(files => {
   console.log(files) // Equals []
 })

function getFiles(folder){
  return recursive(folder, ["*.DS_Store", ignoreFiles])
}

function ignoreFiles(file,stats){
  console.log(file) // Equals C:\Users\Greg\Dropbox\Designs\Valentines Day
  const slash = file.substr(file.lastIndexOf(`/`)+1),
        name = slash.toLowerCase(),
        fExt = name.substr(name.length - 4),
        hoodie = name.includes('hoodie');
  return fExt === '.png' && hoodie === false ? false : true
}

How to ignore protected files during the reading ?

Hi

In the moment when I get an error ( Error: EPERM: operation not permitted, stat 'F:\System Volume Information' ), Files I can't get all others files which are OK.

Is it a way to force to continue to analyze or a way to check after each file what to do ?

Thank you

NB: maybe my question is stupid but in fact, I'm sticked with my idea, I don't know how to get a workaround. Of course I can create again the stuff at the hand but it's not my aim.

OK this: #43 seems near that I want

Supporting directory ignore?

Curious, would you accept a PR for an ignore option? (To ignore, say, node_modules, or any other dir/file you may want)

README contains incorrect?

Hi, thanks helpful package. This is so tiny report but I would like to suggest modify README.

README says:

recursive("some/path", function (err, files) {
  // `files` is an array of absolute file paths
  console.log(files);
});

but path.join() doesn't resolve paths into an absolute path.

var filePath = p.join(path, file);

So if dirpath as argument is relative, the result paths are relative too.
thanks :)

Options object and match

It'd be very useful if we could do this:

// ignore files named 'foo.cs' or files that end in '.html'.
recursive('some/path', {ignore:['foo.cs', '*.html']}, function(err, files) {
  console.log(files);  //-> [file1, file2, file3]
});

and this:

// match only files named 'foo.cs' or files that end in '.html'.
recursive('some/path', {match:['foo.cs', '*.html']}, function(err, files) {
  console.log(files);  //-> [file1, file2, file3]
});

and both combined, of course (running match before ignore).

For backwards compatibility, arguments[1] should still look for the ignore array as well.

Ignore auto-generated folders

Is it possible to ignore fodlers like node_modules or any other auto-generated folders depending on a given regex expression?

RFC: Any plan for next MAJOR ?

The Features which I wanted:

  1. Force to continue when the recursion get error. (In this time, we get nothing when the fs.stat throw error), maybe support options named forceContinue.
  2. Refactor with typescript ?

License

Can you add a LICENSE file for this so I can include it when using this module?

Thanks in advance

Option to limit the maximum path depth

For example, if specifying maxDepth: 2 with path /a/b:

  • /a/b/c/d directory will be fs.readdir-ed
  • /a/b/c/d/file.txt will be returned in files
  • /a/b/c/d/e directory will NOT be fs.readdir-ed

Right now, using ignore will still fs.readdir() all children directories with no depth limit.

Negative matches do not enter folders failing the match

I'm using recursive-readdir to get a list of files with a specific extension. I use '!txt' as macth. This causes folders not having 'txt' in their name to be ignored.
I was a bit surprised by this behaviour, maybe it should be documented, or the match should only be applied to files and not folders.

Multiple negated ignores

This setup will not match anything:

ignore: [
  '*',
  '!readme.md',
  '!history.md'
]

this will work in a .gitignore though, because ! rules only negate anything matched previously.

Doesn't preserve order of files

In a project I am working on, after lengthy troubleshooting, I finally found that recursive-readdir doesn't yield consistent ordering of files by file name. Even though it's not an advertised feature, I think at least it should be documented that order is not guaranteed. Also, implementing the feature shouldn't be hard.

Feature request: Ability to pass a function to ignore

To make the current system for ignoring files much more powerful, it would be great if we could pass a function to ignore instead of just minimatch strings. The function would take in the file path and lstat object, and return true to ignore the file, or false to keep it.

This would add a lot of power to recursive-readdir's ignore system, addressing both the use cases of #14 and my comments on e2f2ea0 in one fell swoop. It would also really help with some of the problems we've discussed in #23.

Cannot call the function twice in a row in visual studio code

Hello

I am using your package to extract the list of files of two folders with the following function in visual studio code 1.28.2 with node version 8.9.3

`
var recursive = require("recursive-readdir");

async function generate() {
var files1 = [] ;
var files = [] ;
await recursive('path1')
.then((files) => files1 = files)
.then(recursive('path2'))
then((files) => files = files)
.catch(e => console.error(e));
}
`

and the following launch configuration

{ "type": "node", "request": "launch", "name": "Generate Report", "cwd": "${workspaceRoot}", "protocol": "inspector", "runtimeArgs": [ "inspect" ], "program": "${workspaceFolder}/test/test.js" }

and the following package.json

'
{
"name": "multiple-cucumber-html-reporter",
"version": "1.3.0",
"description": "Generate beautifull cucumberjs reports for multiple instances (browsers / devices)",
"keywords": [
"cucumber",
"html",
"test report",
"multiple-cucumber-html-reporter",
"html report",
"json to html"
],
"main": "lib/generate-report.js",
"scripts": {
"test": "jasmine",
"unit.test.coverage": "JASMINE_CONFIG_PATH=test/unit/jasmine.json istanbul cover jasmine",
"coveralls": "cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js && rm -rf ./coverage",
"dev": "nodemon server.js",
"test-cover": "NODE_ENV=test nyc --clean --reporter=lcov --reporter=json --reporter=text npm test "
},
"repository": {
"type": "git",
"url": "git+https://github.com/wswebcreation/multiple-cucumber-html-reporter.git"
},
"author": "wswebcreation",
"bugs": {
"url": "https://github.com/wswebcreation/multiple-cucumber-html-reporter/issues"
},
"homepage": "https://github.com/wswebcreation/multiple-cucumber-html-reporter#readme",
"dependencies": {
"adm-zip": "^0.4.7",
"async": "^2.6.1",
"bluebird": "^3.5.2",
"body-parser": "^1.18.2",
"bson": "^3.0.2",
"c21e": "^1.1.5",
"chalk": "2.4.1",
"cucumber-messages": "^2.0.0",
"date-and-time": "^0.6.3",
"ejs": "^2.5.8",
"express": "^4.16.3",
"find": "^0.2.9",
"fs-extra": "^7.0.0",
"fs-promise": "^2.0.3",
"gherkin": "^5.1.0",
"helmet-csp": "^2.7.0",
"js-base64": "^2.4.9",
"json-future": "^2.1.3",
"jsonfile": "5.0.0",
"jszip": "^3.1.5",
"lodash": "^4.17.11",
"minimist": "^1.2.0",
"minipass": "^2.3.5",
"mkdir": "0.0.2",
"mkdirp": "^0.5.1",
"mongodb": "^3.0.5",
"opn": "^5.4.0",
"proxyquire": "^2.1.0",
"recursive-readdir": "^2.2.2",
"safe-buffer": "^5.1.2",
"tar": "^4.4.1",
"util": "^0.11.0",
"uuid": "^3.3.2",
"winston": "^3.1.0",
"yallist": "^3.0.2"
},
"devDependencies": {
"coveralls": "^3.0.2",
"istanbul": "0.4.5",
"jasmine": "^3.3.0",
"jasmine-console-reporter": "^3.1.0",
"jasmine-expect": "^3.8.4",
"nodemon": "^1.18.5",
"nyc": "^13.1.0",
"supertest": "^3.3.0"
},
"directories": {
"doc": "docs",
"lib": "lib",
"test": "test"
},
"license": "ISC"
}
`

and the first call to the recursive function is performed correctly, but the second one doesn't provide any output and makes the execution to finish intermediately.

Should I use the function in any other way in order to be able to use it twice in a row?

Regards

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.