lyonsyonii / hunt-rs Goto Github PK

When using -ss, the results are still delivered as one batch after a few seconds if searching my whole system. For piping to a fuzzy finder, it would be handy if the results were posted incrementally with that option.

Greetings @LyonSyonII Liam, sending peace.

I write because I have been using your hunt-rs project/tool to find files on a Linux file system. Thank you. It worked so well and so fast, it inspired me to try and learn Rust.

I write to request 2 features to your script around file duplication. This will help to round out the feature set.

1. The script should return a list of unique files, i.e. the code does a hash/digest on each file and insert it into a HashMap along with the path. It would also ignore the next file match.

2. The script will also keep track of a list of duplicated files found and also display these.

You can handle this use case under an option like --duplicates or --unique as an example.

The use cases are:

1. Virtual environments and containerization (python, java scripts, docker), tends to copy or download multiple copies of the same files, which are sometimes left abandoned.

2. Installation of multiple similar software tend to duplicate their config files. For example, Nginx and Apache my have duplicate in config, certs, key, and security settings.

3. Admin simply needs to know.

Of course, duplicated files take up unnecessary space. But more importantly, if these files are left for too long without being updated, they could potentially be a security risk.

One Love.

Firstly, I wanted to say the performance of hunt is outstanding. What took me 2 minutes 8 seconds in find, took 10 seconds in hunt. You should be getting more stars.

The issue I have is with the simple output displaying the full path of the filename when a full path was not described in the cli command.
In other words, please have hunt auto-detect explicit path or relative path usage in the cli command and adjust the results output accordingly to use the same path type(relative path or full explicit path).

find using a relative path(the dot .). Please notice the result also uses a relative path since it auto-detected it:

time find . -type f -name "SCP_173 -print > ../allFindResults.txt
./rooms/room-999/SCP_173

hunt using a relative path(the dot .). Please notice the result contained full explicit path, but that was undesired since the cli command never used a full explicit path. Also note it creates much bigger results since each line in the result has a longer full path.

time hunt -c -h -t f -s -e SCP_173 . > ../allHuntExactResults.txt
/mnt/someremotedrive/a/b/c/d/thispathcouldbeverylong/andtheresultfilewillbemuchlarger/rooms/room-999/SCP_173

The change request is to achieve a similar looking result file and exact same file size as find, but using hunt.

time hunt -c -h -t f -s -e SCP_173 . > ../allHuntExactResults.txt
./rooms/room-999/SCP_173

Thank you for listening.

Can you add and specify which license hunt is available under?

Description

As I've started to bring the latest version of hunt into the AUR, I've happened to get some linking errors on my machine. What's more weird, it happened only for me while running cargo build within makepkg enviroment, which could explain it went unnoticed for quite some time.

I've linked the issue to the lack of mimalloc in the linker command-line while building it with LTO enabled – in makepkg, this is done simply by appending lto to the options=() array in the makepkg.conf or in package-specific options. I currently have no clue what LTO changes in terms of generating a command for linking the native libraries into the final binary, I didn't dig into this problem that much. So there might be a better proper way to fix it, maybe it's even an issue for an upstream project to resolve in the reality.

Logs

I'll skip the part that mentions exactly which symbols aren't being found by the linker, to keep the logs a bit shorter than they would be normally, since I've already found the right native library that provides them (symbols in logs quite point to mimalloc with the errors telling there's undefined reference to symbols like mi_free or mi_malloc_aligned). I've also splitted the logs by the beginning part and the ending part.

Here's what's at start of the logs (basically the command line used for linting this, with some stuff replaced to not to leak any info about some paths on my PC or my username):

error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="[REDACTED]" VSLANG="1033" "cc" "-m64" "/tmp/rustcIiP3az/symbols.o" "[REPODIR]/hunt-2.1.0/target/release/deps/hunt-9c727d0082a78a8e.hunt.5e3c86399b7808a-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "[REPODIR]/hunt-2.1.0/target/release/deps" "-L" "[REPODIR]/hunt-2.1.0/target/release/build/libmimalloc-sys-d2896777ded63943/out" "-L" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcIiP3az/liblibmimalloc_sys-fdda0fd318ea1a6d.rlib" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-8e138eaf26ebb4a8.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/[USER]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "[REPODIR]/hunt-2.1.0/target/release/deps/hunt-9c727d0082a78a8e" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"

Here's useful part and the last line of the logs I get from cargo, it's basically telling how the one might resolve this in their project (I used that to go with the solution utilising the build.rs):

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `hunt` (bin "hunt") due to previous error

Workarounds

I was able to workaround the error by enforcing to link mimalloc with the following build.rs (placed in the top-level directory of the Rust project, i.e. next to Cargo.toml):

fn main() {
    println!("cargo:rustc-link-lib=mimalloc");
}

I suppose claiming that LTO is unsupported is another way to resolve this issue, so either action could be done. As mentioned before, I'm myself the current maintainer of hunt in AUR, so I currently went for using the build.rs as a workaround, but an option to disable LTO for this package is possible as well.