Giter Club home page Giter Club logo

find-remove's Introduction

find-remove

Build Status

recursively finds files by filter options from a start directory onwards and deletes only those which meet conditions you can define. useful if you want to clean up a directory in your node.js app.

you can filter by extensions, names, level in directory structure, file creation date and ignore by name, yeah!

installation

to install find-remove, use npm:

$ npm install -S find-remove

then in your node.js app, get reference to the function like that:

const findRemoveSync = require('find-remove')

quick examples

1. delete all _.bak or _.log files within the /temp/ directory

const result = findRemoveSync('/temp', { extensions: ['.bak', '.log'] })

the return value result is a json object with successfully deleted files. if you output result to the console, you will get something like this:

{
    '/tmp/haumiblau.bak': true,
    '/tmp/dump.log': true
}

2. delete all files called 'dump.log' within the /temp/ directory and within its subfolders

var result = findRemoveSync('/temp', { files: 'dump.log' })

3. same as above, but also deletes any subfolders

var result = findRemoveSync('/temp', { files: 'dump.log', dir: '*' })

4. delete all *.bak files but not file 'haumiblau.bak'

var result = findRemoveSync('/temp', { extensions: ['.bak'], ignore: 'haumiblau.bak' })

5. delete recursively any subdirectory called 'CVS' within /dist/

var result = findRemoveSync('/dist', { dir: 'CVS' })

6. delete all jpg files older than one hour with limit of 100 files deletion per operation

var result = findRemoveSync('/tmp', {
  age: { seconds: 3600 },
  extensions: '.jpg',
  limit: 100
})

7. delete all files with prefix 'filenamestartswith'

var result = findRemoveSync('/tmp', { prefix: 'filenamestartswith' })

8. apply filter options only for two levels inside the /temp directory for all tmp files

var result = findRemoveSync('/tmp', { maxLevel: 2, extensions: '.tmp' })

this deletes any .tmp files up to two levels, for example: /tmp/level1/level2/a.tmp

but not /tmp/level1/level2/level3/b.tmp

why the heck do we have this maxLevel option? because of performance. if you care about deep subfolders, apply that option to get a speed boost.

9. delete everything recursively (hey, who needs that when you can use nodejs' fs.unlink?)

var result = findRemoveSync(rootDirectory, { dir: '*', files: '*.*' })

10. delete all files that match a regular expression

var result = findRemoveSync(rootDirectory, { files: 'example[1-3]', regex: true })

this deletes files example1.txt, example2.txt, and example3.txt, but not example8.txt.

11. delete all directories that match a regular expression

var result = findRemoveSync(rootDirectory, { dir: '^assets_', regex: true })

this deletes all directories that start with assets_.

api

findRemoveSync(dir, options)

findRemoveSync takes any start directory and searches files from there for removal. the selection of files for removal depends on the given options. and at last, it deletes the selected files/directories.

arguments

  • dir - any directory to search for files and/or directories for deletion (does not delete that directory itself)
  • options - currently those properties are supported:
    • files - can be a string or an array of files you want to delete within dir.
    • dir - can be a string or an array of directories you want to delete within dir.
    • extensions - this too, can be a string or an array of file extentions you want to delete within dir.
    • ignore - useful to exclude some files. again, can be a string or an array of file names you do NOT want to delete within dir
    • age.seconds - can be any float number. findRemoveSync then compares it with the file stats and deletes those with modification times older than age.seconds
    • limit - can be any integer number. Will limit the number of files to be deleted at single operation to be limit
    • prefix - can be any string. Will delete any files that start with prefix.
    • maxLevel - advanced: limits filtering to a certain level. useful for performance. recommended for crawling huge directory trees.
    • test - advanced: set to true for a test run, meaning it does not delete anything but returns a JSON of files/directories it would have deleted. useful for testing.
    • regex - set to true to treat files or dir option strings as regular expression patterns.

as a precaution, nothing happens when there are no options.

the unit tests are good examples on how to use the above arguments.

returns

JSON of files/directories that were deleted. For limit option - will only return number of files deleted.

todo

  • needs a rewrite
  • add more filtering options (e.g. combinations)
  • have an asynchronous solution
  • use streams instead

license

MIT

find-remove's People

Contributors

binarykitchen avatar dependabot[bot] avatar nrullo avatar pbakondy avatar pzhine avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

find-remove's Issues

Node 16 support

Hi,

thanks a lot for this package.

While upgrading some projects to node 16 (will be the next LTS version) I noticed that this package claims to not be compatible:
error [email protected]: The engine "node" is incompatible with this module. Expected version "^14.4.0". Got "16.0.0" error Found incompatible module.

I have not yet tested if everything would still run when installed with --ignore-engines. The installation itself went fine, however.

Best,
Michael

Node 18 support

Hi,

thanks a lot for this package.

While upgrading some projects to node 18 (will be the next LTS version) I noticed that this package claims to not be compatible:

npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '^16.13.1', npm: '^8.1.2' },
npm WARN EBADENGINE   current: { node: 'v18.9.1', npm: '8.19.2' }
npm WARN EBADENGINE }

Nevertheless in my project by installing with --ignore-engines seems to work well.

Best,
Michael

Odd behavior with limit

const result = findRemoveSync(process.env.LOG_BASE_DIR, {
  age: { seconds: process.env.SECONDS_TO_RETAIN_LOG_FILES },
  extensions: ".log",
  limit: 100
});

the base dir has multiple directories under it and the log files are under those.
If I have counted right there are around 94 files scattered amongst sub-folders, all *.log, definitely not many more than 100 if at all. All old than the 20s set and it seems to take 2 goes to delete all the files.

I have checked around to see that none of the files are open anywhere. Is there anything else that I may be doing wrong that may be causing this?
What is the behavior if it fails to delete one file, will it still go on to delete the others?

Thanks

return value is never correct

the returned array is always empty.
due to it's declared as an array (line 128):
var removed = [];
however when adding filePath to it, it's used as a dictionary (line 167):
removed[currentFile] = true;

this is... such an obvious bug..
i'm shocked it has never been fixed in so many releases...

Awesome, Async support please!

Great stuff, love cleaning up my all temp files of x age with a 1 liner :)

Async support would be great, but otherwise one happy user here.

`EBADENGINE` warning shows up when installing the find-remove package

When I installed the package, there is a warning message:

npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '^16.1.0', npm: '^7.1.0' },
npm WARN EBADENGINE   current: { node: 'v16.13.0', npm: '8.1.3' }
npm WARN EBADENGINE }

I think that the npm version required can be writed as >=7.1.0, but I am not sure. Please help.

Remove folders with name pattern?

I want to remove all folders and their content within TEMP that start with "scoped".
var result = findRemoveSync(process.env.TMP, { dir: "scoped_dir*" })
doesn't work

Age causes permission denied error

Compare these two calls:

FINDREMOVE(OS.tmpdir(), { extensions: [ '.pdf' ], maxLevel: 2 })
Returns an empty object {}

FINDREMOVE(OS.tmpdir(), { extensions: [ '.pdf' ], maxLevel: 2, age: { seconds: 1800 } })

Throws an exception:
Error: EACCES: permission denied, unlink '/tmp/.log-publish-records/_var_log_aws-sqsd_default.log-20180826.gz'
at Error (native)
at Object.fs.unlinkSync (fs.js:1104:18)
at rimrafSync (/var/app/current/node_modules/rimraf/rimraf.js:297:17)
at /var/app/current/node_modules/rimraf/rimraf.js:332:5
at Array.forEach (native)
at rmkidsSync (/var/app/current/node_modules/rimraf/rimraf.js:331:26)
at rmdirSync (/var/app/current/node_modules/rimraf/rimraf.js:324:7)
at Function.rimrafSync [as sync] (/var/app/current/node_modules/rimraf/rimraf.js:295:9)
at module.exports (/var/app/current/node_modules/find-remove/find-remove.js:176:28)
at /var/app/current/node_modules/find-remove/find-remove.js:157:34

It seems that by adding the { age } property the module tries to delete files that it wouldn't otherwise care about.

Expected behaviour: Adding { age } property further limits the find based on the age of the files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.