christopherschultz / fast-file-count Goto Github PK

Amazing work. Glad you crushed it. I removed the debug lol but I ripped it up a little bit because I wanted to count all folders in each sub directory. Worked flawless and its about 2x as fast as my find with sub find loops in bash(way more code on yours but no big deal!)

opendir does that guarantee any sort order? I was thinking id just read in the time then sort at the end as it should not effect performance really to get the extra info. Any other thoughts? Is there an innate function that determines read order?

The initial directory listing (ls or find etc) is nearly an instant result its a SMB share so to get all the files and details was what really took longer. I can with out performance do it via bash but....just looking if you had any real slick ideas besides the methods I thought. (I only need that top level date).

A

Hey, I've found this utility very useful and thought it might make a good addition to Homebrew. Would you have any qualms with me submitting this to brew as a new formula?

Hi Chris,

Would it be possible to add count of files and count of subdirs + size recursively ?
To get something like:

path ; #localsubfiles ; #localsubdirs ; localsize ; #totalsubfiles ; #totalsubdirs ; totalsize
dir1;0;10;0;5345;701;45606003
dir1/dir11;10;3;133342;234;28;345660
dir1/dir12;1;6;42;23657;333;35660
...

Regards

This thing counted ~8M files in no time! Thanks!

We regularly deal with very large NFS-based directories. These are on very, very fast NFS servers (multiple 100Gbps connections) serving thousands of clients and regularly hit >500K IOPS.

For a directory tree with 2,950,916 files and 232,782 directories, this took:

real 23m20.367s
user 0m2.605s
sys 1m26.131s

I had to delete a directory tree with 40M files yesterday so 3M isn't even all that large here.