stefanoj3 / dirstalk Goto Github PK

Modern alternative to dirbuster/dirb

License: MIT License

Makefile 2.07% Go 93.14% Dockerfile 0.40% Shell 4.40%

enumeration dirbuster dirb url-bruteforcer dictionaries bruteforce bruteforce-wordlist web-content-scanner auditing security pentest hacktoberfest

dirstalk's Introduction

Dirstalk

Dirstalk is a multi threaded application designed to brute force paths on web servers.

The tool contains functionalities similar to the ones offered by dirbuster and dirb.

Here you can see it in action:

How to use it
Download
Development
License

↑ How to use it

The application is self-documenting, launching dirstalk -h will return all the available commands with a short description, you can get the help for each command by doing distalk <command> -h.

EG dirstalk result.diff -h

Scan

To perform a scan you need to provide at least a dictionary and a URL:

dirstalk scan http://someaddress.url/ --dictionary mydictionary.txt

As mentioned before, to see all the flags available for the scan command you can just call the command with the -h flag:

dirstalk scan -h

Example of how you can customize a scan:

dirstalk scan http://someaddress.url/ \
--dictionary mydictionary.txt \
--http-methods GET,POST \
--http-timeout 10000 \
--scan-depth 10 \
--threads 10 \
--socks5 127.0.0.1:9150 \
--cookie name=value \
--use-cookie-jar \
--user-agent my_user_agent \
--header "Authorization: Bearer 123"

Currently available flags:

      --cookie stringArray             cookie to add to each request; eg name=value (can be specified multiple times)
  -d, --dictionary string              dictionary to use for the scan (path to local file or remote url)
      --header stringArray             header to add to each request; eg name=value (can be specified multiple times)
  -h, --help                           help for scan
      --http-cache-requests            cache requests to avoid performing the same request multiple times within the same scan (EG if the server reply with the same redirect location multiple times, dirstalk will follow it only once) (default true)
      --http-methods strings           comma separated list of http methods to use; eg: GET,POST,PUT (default [GET])
      --http-statuses-to-ignore ints   comma separated list of http statuses to ignore when showing and processing results; eg: 404,301 (default [404])
      --http-timeout int               timeout in milliseconds (default 5000)
      --out string                     path where to store result output
      --scan-depth int                 scan depth (default 3)
      --socks5 string                  socks5 host to use
  -t, --threads int                    amount of threads for concurrent requests (default 3)
      --use-cookie-jar                 enables the use of a cookie jar: it will retain any cookie sent from the server and send them for the following requests
      --user-agent string              user agent to use for http requests

Useful resources

here you can find dictionaries that can be used with dirstalk
tordock is a containerized Tor SOCKS5 that you can use easily with dirstalk (just docker run -d -p 127.0.0.1:9150:9150 stefanoj3/tordock:latest and then when launching a scan specify the following flag: --socks5 127.0.0.1:9150)

Dictionary generator

Dirstalk can also produce it's own dictionaries, useful for example if you want to check if a specific set of files is available on a given web server.

Example:

dirstalk dictionary.generate /path/to/local/files --out mydictionary.txt

The result will be printed to the stdout if no out flag is specified.

↑ Download

You can download a release from here or you can use a docker image. (eg docker run stefanoj3/dirstalk dirstalk <cmd>)

If you are using an arch based linux distribution you can fetch it via AUR: https://aur.archlinux.org/packages/dirstalk/

Example:

yay -S aur/dirstalk

↑ Development

All you need to do local development is to have make and golang available and the GOPATH correctly configured.

Then you can just clone the project, enter the folder and:

make dep                                     # to fetch dependencies
make tests                                   # to run the test suite
make check                                   # to check for any code style issue
make fix                                     # to automatically fix the code style using goimports
make build                                   # to build an executable for your host OS (not tested under windows)

make help

will print a description of every command available in the Makefile.

Wanna add a functionality? fix a bug? fork and create a PR.

↑ Plans for the future

Add support for rotating SOCKS5 proxies
Scan a website pages looking for links to bruteforce
Expose a webserver that can be used to launch scans and check their status
Introduce metrics that can give a sense of how much of the dictionary was found on the remote server

dirstalk's People

Contributors

Stargazers

Watchers

dirstalk's Issues

Evaluate if gosec can easily be integrated in CI

https://github.com/securego/gosec

scan codebase with malligned

scan codebase with https://github.com/mdempsky/maligned and see if there be any improvement for memory usage

Add support for calling a custom command for each result found

Makefile release* commands should use a reproducible environment

https://hub.docker.com/r/goreleaser/goreleaser/tags

dictionary

create dictionary based of:

https://pastebin.com/LHJrNpnV
https://github.com/danielmiessler/SecLists/tree/master/Discovery/Web-Content (raft)
add entries from common open source projects

dial tcp: lookup site.com on [::1]:53: read udp [::1]:58403->[::1]:53: read: connection refused" method=GET path="~user3" DEBU terminating worker: producer channel closed ERRO failed to perform request depth=3

what is your solution with this?

scan output to file

add possibility to save the result to a file
add command to compare 2 results and print out diff (useful for automated scans)

add cewl example to readme

https://github.com/digininja/CeWL

dictionary generation from html

Introduce possibility to generate dictionary out of html files like https://manpages.ubuntu.com/manpages/bionic/man1/html2dic.1.html

regexp filter

Add possibility to filter results based on content body and regexp

Add support for prefixes and suffixes to be added to each word of the dictionary when scanning

result screenshot

https://github.com/mafredri/cdp
https://godoc.org/github.com/tebeka/selenium
https://hacks.mozilla.org/2017/12/using-headless-mode-in-firefox/

could use headless chrome or selenium to integrate and capture screenshots of all the results found and create a nice html page to view them

Add support for specifying a custom UA

Asciinema & update readme

Update readme with the latest functionalities (cookie jar, user agent, headers)
Add Asciinema example of dirstalk usage

stricter linter

Make golangci-lint stricter by enabling more linters and fix all the reported issues.

EG golangci-lint run --enable-all

dictionary error

Hello,

When I try to run a scam, this is what I get. I tried moving into the directory and calling the file and it has been to no avail.

docker run stefanoj3/dirstalk dirstalk scan "https://example.com/" -d "/usr/share/dirbuster/wordlists/directory-list-1.0.txt" Error: failed to build dictionary: dictionary: failed to get/usr/share/dirbuster/wordlists/directory-list-1.0.txt`: Get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: unsupported protocol scheme ""
Usage:
dirstalk scan [url] [flags]

Flags:
--cookie stringArray cookie to add to each request; eg name=value (can be specified multiple times)
-d, --dictionary string dictionary to use for the scan (path to local file or remote url)
--header stringArray header to add to each request; eg name=value (can be specified multiple times)
-h, --help help for scan
--http-cache-requests cache requests to avoid performing the same request multiple times within the same scan (EG if the server reply with the same redirect location multiple times, dirstalk will follow it only once) (default true)
--http-methods strings comma separated list of http methods to use; eg: GET,POST,PUT (default [GET])
--http-statuses-to-ignore ints comma separated list of http statuses to ignore when showing and processing results; eg: 404,301 (default [404])
--http-timeout int timeout in milliseconds (default 5000)
--out string path where to store result output
--scan-depth int scan depth (default 3)
--socks5 string socks5 host to use
-t, --threads int amount of threads for concurrent requests (default 3)
--use-cookie-jar enables the use of a cookie jar: it will retain any cookie sent from the server and send them for the following requests
--user-agent string user agent to use for http requests

Global Flags:
-v, --verbose verbose mode

level=fatal msg="Execution error" err="failed to build dictionary: dictionary: failed to get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: Get /usr/share/dirbuster/wordlists/directory-list-1.0.txt: unsupported protocol scheme """
`

sigint / ctr-c

handle signint: do not jus tkill the application, stop the scan and print the summary of what was found so far

provide blackarch package

https://blackarch.org/tools.html

failed to build dictionary: dictionary

Hello @stefanoj3 ,

When running the below command, specifically placing single quotes in the 2nd to last flag I got the aforementioned error., Interestingly if I removed the quotes it worked and later in testing i was able to get it to successfully run with the single quotes. Its not a major issue but am raising it so you know it exists

docker run stefanoj3/dirstalk dirstalk scan http://example.com -d "https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt" --no-check-certificate --http-statuses-to-ignore '404,301' -t 10
Error: failed to build dictionary: dictionary: failed to get `https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt`: Get https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Usage:
  dirstalk scan [url] [flags]

Flags:
      --cookie stringArray             cookie to add to each request; eg name=value (can be specified multiple times)
  -d, --dictionary string              dictionary to use for the scan (path to local file or remote url)
      --header stringArray             header to add to each request; eg name=value (can be specified multiple times)
  -h, --help                           help for scan
      --http-cache-requests            cache requests to avoid performing the same request multiple times within the same scan (EG if the server reply with the same redirect location multiple times, dirstalk will follow it only once) (default true)
      --http-methods strings           comma separated list of http methods to use; eg: GET,POST,PUT (default [GET])
      --http-statuses-to-ignore ints   comma separated list of http statuses to ignore when showing and processing results; eg: 404,301 (default [404])
      --http-timeout int               timeout in milliseconds (default 5000)
      --no-check-certificate           to skip checking the validity of SSL certificates
      --out string                     path where to store result output
      --scan-depth int                 scan depth (default 3)
      --socks5 string                  socks5 host to use
  -t, --threads int                    amount of threads for concurrent requests (default 3)
      --use-cookie-jar                 enables the use of a cookie jar: it will retain any cookie sent from the server and send them for the following requests
      --user-agent string              user agent to use for http requests

Global Flags:
  -v, --verbose   verbose mode

level=fatal msg="Execution error" err="failed to build dictionary: dictionary: failed to get `https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt`: Get https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"

Add support to specify any number of custom headers to the http calls

Certificate signed by Unknown

Hello,

It seems to fail if it doesn't trust the cert from the target web server. Below is sample output from when I ran it. Is there anyway a -k (curl) or --no-check-certificate (wget) flag can be added?

docker run stefanoj3/dirstalk dirstalk scan "https://redacted.example.com" -d https://raw.githubusercontent.com/daviddias/node-dirbuster/master/lists/directory-list-1.0.txt
level=info msg="Starting scan" cookie-jar=false cookies= dictionary-length=141694 headers= scan-depth=3 socks5="<nil>" threads=3 timeout=5000 url="https://redacted.example.com" user-agent=
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/cgi-bin: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=cgi-bin
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/education: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=education
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/betsie: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=betsie
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/accessibility: x509: certificate signed by unknown authority" method=GET path=accessibility
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/go: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=go
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/accesskeys: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=accesskeys
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/-: x509: certificate signed by unknown authority" method=GET path=-
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/toolbar: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=toolbar
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/radio: x509: certificate signed by unknown authority" method=GET path=radio
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/tv: x509: certificate signed by unknown authority" method=GET path=tv
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/whereilive: x509: certificate signed by unknown authority" method=GET path=whereilive
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/talk: x509: certificate signed by unknown authority" method=GET path=talk
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/homepage: x509: certificate signed by unknown authority" method=GET path=homepage
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/a-z: x509: certificate signed by unknown authority" method=GET path=a-z
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/bb: x509: certificate signed by unknown authority" method=GET path=bb
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/int: x509: certificate signed by unknown authority" method=GET path=int
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/textonly: x509: certificate signed by unknown authority" method=GET path=textonly
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/oth: x509: certificate signed by unknown authority" method=GET path=oth
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/mobile: x509: certificate signed by unknown authority" method=GET path=mobile
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/t: x509: certificate signed by unknown authority" method=GET path=t
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/help: x509: certificate signed by unknown authority" method=GET path=help
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/search: x509: certificate signed by unknown authority" method=GET path=search
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/pl1: x509: certificate signed by unknown authority" method=GET path=pl1
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/makehomepage: x509: certificate signed by unknown authority" method=GET path=makehomepage
level=error msg="failed to perform request" depth=3 error="Get https://redacted.example.com/text: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" method=GET path=text

scripting target processor

Could use otto as a scripting language to add more complex processing rules for dictionaries

scan result feature

add flag to specify file where to store a scan result
add command to view scan results
~~add flag to execute another command for each scan result~~ canceled
add command to diff between 2 scan results

Docker stopping issue

Hello @stefanoj3 , we meet again lol!

Humor aside, when I ran dirstalk in docker i noticed if i tried do to either ctrl+z or ctrl+c it didnt stop the program but instead moved to the next procedural phase. I had to hit it a few times to eventually make it stop. I am not sure if you were aware but I figured I let you know. Its a minor issue but ya.

Add support for cookie jar in http client

can decide whether or not to retain some cookies
can add cookies for every request - value can now be added to the jar, consider having a "null jar" for cases when the normal cookie jar is not provided (the default behavior)

stefanoj3 / dirstalk Goto Github PK

dirstalk's Introduction

Dirstalk

Contents

↑ How to use it

Scan

Example of how you can customize a scan:

Currently available flags:

Useful resources

Dictionary generator

Example:

↑ Download

↑ Development

↑ Plans for the future

dirstalk's People

Contributors

Stargazers

Watchers

Forkers

dirstalk's Issues

Recommend Projects

Recommend Topics

Recommend Org