Giter Club home page Giter Club logo

massdns's Introduction

MassDNS

A high-performance DNS stub resolver

MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amount of domain names in the order of millions or even billions. Without special configuration, MassDNS is capable of resolving over 350,000 names per second using publicly available resolvers.

Contributors

Compilation

Clone the git repository and cd into the project root folder. Then run make to build from source. If you are not on Linux, run make nolinux. On Windows, the Cygwin packages gcc-core, git and make are required.

Usage

Usage: ./bin/massdns [options] [domainlist]
  -b  --bindto           Bind to IP address and port. (Default: 0.0.0.0:0)
      --busy-poll        Use busy-wait polling instead of epoll.
  -c  --resolve-count    Number of resolves for a name before giving up. (Default: 50)
      --drop-group       Group to drop privileges to when running as root. (Default: nogroup)
      --drop-user        User to drop privileges to when running as root. (Default: nobody)
      --extended-input   Input names are followed by a space-separated list of resolvers.
                         These are used before falling back to the resolvers file.
      --filter           Only output packets with the specified response code.
      --flush            Flush the output file whenever a response was received.
  -h  --help             Show this help.
      --ignore           Do not output packets with the specified response code.
  -i  --interval         Interval in milliseconds to wait between multiple resolves of the same
                         domain. (Default: 500)
  -l  --error-log        Error log file path. (Default: /dev/stderr)
      --norecurse        Use non-recursive queries. Useful for DNS cache snooping.
  -o  --output           Flags for output formatting.
      --predictable      Use resolvers incrementally. Useful for resolver tests.
      --processes        Number of processes to be used for resolving. (Default: 1)
  -q  --quiet            Quiet mode.
      --rand-src-ipv6    Use a random IPv6 address from the specified subnet for each query.
      --rcvbuf           Size of the receive buffer in bytes.
      --retry            Unacceptable DNS response codes.
                         (Default: All codes but NOERROR or NXDOMAIN)
  -r  --resolvers        Text file containing DNS resolvers.
      --root             Do not drop privileges when running as root. Not recommended.
  -s  --hashmap-size     Number of concurrent lookups. (Default: 10000)
      --sndbuf           Size of the send buffer in bytes.
      --status-format    Format for real-time status updates, json or ansi (Default: ansi)
      --sticky           Do not switch the resolver when retrying.
      --socket-count     Socket count per process. (Default: 1)
  -t  --type             Record type to be resolved. (Default: A)
      --verify-ip        Verify IP addresses of incoming replies.
  -w  --outfile          Write to the specified output file instead of standard output.

Output flags:
  L - domain list output
  S - simple text output
  F - full text output
  B - binary output
  J - ndjson output

Advanced flags for the domain list output mode:
  0 - Include NOERROR replies without answers.

Advanced flags for the simple output mode:
  d - Include records from the additional section.
  i - Indent any reply record.
  l - Separate replies using a line feed.
  m - Only output reply records that match the question name.
  n - Include records from the answer section.
  q - Print the question.
  r - Print the question with resolver IP address, Unix timestamp and return code prepended.
  s - Separate packet sections using a line feed.
  t - Include TTL and record class within the output.
  u - Include records from the authority section.

Advanced flags for the ndjson output mode:
  e - Write a record for each terminal query failure.

For a detailed description of the command line interface, please consult the man page using man ./doc/massdns.1.

Example

Resolve all AAAA records from domains within domains.txt using the resolvers within resolvers.txt in lists and store the results within results.txt:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA domains.txt > results.txt

This is equivalent to:

$ ./bin/massdns -r lists/resolvers.txt -t AAAA -w results.txt domains.txt

Example output

By default, MassDNS will output response packets in text format which looks similar to the following:

;; Server: 77.41.229.2:53
;; Size: 93
;; Unix time: 1513458347
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51298
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
example.com. IN A

;; ANSWER SECTION:
example.com. 45929 IN A 93.184.216.34

;; AUTHORITY SECTION:
example.com. 24852 IN NS b.iana-servers.net.
example.com. 24852 IN NS a.iana-servers.net.

The resolver IP address is included in order to make it easier for you to filter the output in case you detect that some resolvers produce bad results.

Resolving

The repository includes the file resolvers.txt consisting of a filtered subset of the resolvers provided by the subbrute project. Please note that the usage of MassDNS may cause a significant load on the used resolvers and result in abuse complaints being sent to your ISP. Also note that the provided resolvers are not guaranteed to be trustworthy. The resolver list is currently outdated with a large share of resolvers being dysfunctional.

MassDNS's custom, malloc-free DNS implementation currently only supports the most common records. You are welcome to help changing this by collaborating.

PTR records

MassDNS includes a Python script allowing you to resolve all IPv4 PTR records by printing their respective queries to the standard output.

$ ./scripts/ptr.py | ./bin/massdns -r lists/resolvers.txt -t PTR -w ptr.txt

Please note that the labels within in-addr.arpa are reversed. In order to resolve the domain name of 1.2.3.4, MassDNS expects 4.3.2.1.in-addr.arpa as input query name. As a consequence, the Python script does not resolve the records in an ascending order which is an advantage because sudden heavy spikes at the name servers of IPv4 subnets are avoided.

Reconnaissance by brute-forcing subdomains

Perform reconnaissance scans responsibly and adjust the -s parameter to not overwhelm authoritative name servers.

Similar to subbrute, MassDNS allows you to brute force subdomains using the included subbrute.py script:

$ ./scripts/subbrute.py example.com lists/names.txt | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

As an additional method of reconnaissance, the ct.py script extracts subdomains from certificate transparency logs by scraping the data from crt.sh:

$ ./scripts/ct.py example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt

The files names.txt and names_small.txt, which have been copied from the subbrute project, contain names of commonly used subdomains. Also consider using Jason Haddix' subdomain compilation with over 1,000,000 names or the Assetnote wordlist with over 9,000,000 million names.

MassDNS also includes a recon.py wrapper script (beta status) in the scripts folder, which performs subdomain enumeration against authoritative name servers directly and thus does not require third-party resolvers. The concurrency is determined automatically by MassDNS and supports hundreds of thousands of queries per second, while delivering reliable results. On a cheap dedicated server, the Assetnode wordlist can be enumerated in less than a minute. A current limitation is that zone delegation is only handled up to the delegation point. For example, if example.org is enumerated and sub.example.org is delegated to another name server, abc.sub.example.org will not be found by this script if abc.sub is contained in the word list. However, the script will report this fact as ?.sub.example.org in this case.

$ ./scripts/recon.py -d google.com -l lists/best-dns-wordlist.txt > google.txt

Screenshots

Screenshot

Security

MassDNS does not require root privileges and will therefore drop privileges to the user called "nobody" by default when being run as root. If the user "nobody" does not exist, MassDNS will refuse execution. In this case, it is recommended to run MassDNS as another non-privileged user. The privilege drop can be circumvented using the --root argument which is not recommended. Also note that other branches than master should not be used in production at all.

Practical considerations

Performance tuning

MassDNS is a simple single-threaded application designed for scenarios in which the network is the bottleneck. It is designed to be run on servers with high upload and download bandwidths. Internally, MassDNS makes use of a hash map which controls the concurrency of lookups. Setting the size parameter -s hence allows you to control the lookup rate. If you are experiencing performance issues, try adjusting the -s parameter in order to obtain a better success rate.

Rate limiting evasion

In case rate limiting by IPv6 resolvers is a problem, you can make use of --rand-src-ipv6 <your_ipv6_prefix>. MassDNS will then use a raw socket for sending and receiving DNS packets and randomly pick a source IPv6 address from the specified prefix for each query. This requires that MassDNS is run with CAP_NET_RAW privileges. When making use of this method, you should have iptables or nftables drop the DNS traffic received by MassDNS such that no ICMP Port unreachable responses are generated by the operating system, e.g. using ip6tables -p udp --sport 53 -I INPUT -j DROP. Note that this rule is just examplary and would drop all DNS traffic, including traffic for other applications. You might want to adapt the rule to be more fine-grained to fit your use case.

Result authenticity

If the authenticity of results is highly essential, you should not rely on the included resolver list. Instead, set up a local unbound resolver and supply MassDNS with its IP address. In case you are using MassDNS as a reconnaissance tool, you may wish to run it with the default resolver list first and re-run it on the found names with a list of trusted resolvers in order to eliminate false positives.

In case you are enumerating subdomains for a single name, e.g. for example.com, you may want to simply leave out third-party resolvers. In this case, you can directly probe the authoritative nameservers like so:

$ ./bin/massdns -r <(./scripts/auth-addrs.sh example.com) --norecurse -o Je example-com-subdomains.txt > results.txt

Todo

  • Prevent flooding resolvers which are employing rate limits or refusing resolves after some time
  • Implement bandwidth limits
  • Employ cross-resolver checks to detect DNS poisoning and DNS spam (e.g. Level 3 DNS hijacking)
  • Add wildcard detection for reconnaissance
  • Improve reconnaissance reliability by adding a mode which re-resolves found domains through a list of trusted (local) resolvers in order to eliminate false positives
  • Detect optimal concurrency automatically
  • Parse the command line properly and allow the usage/combination of short options without spaces

massdns's People

Contributors

aronmolnar avatar blechschmidt avatar brandonkovacs avatar chenrui333 avatar davift avatar fyoorer avatar h4r5h1t avatar hrbrmstr avatar infosec-it-init avatar istefo avatar janmasarik avatar jmdoubleu avatar leandropereiradacruz avatar marmelatze avatar melvinsh avatar mzpqnxow avatar peterdavehello avatar quirins avatar srknzl avatar thetechromancer avatar zirngibl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

massdns's Issues

only NXDOMAIN scan??

hello . thanks for this program... With massdns i just want to get nxdomain results? What command should I run to find nxdomain subdomain names?

JSON output contains invalid escape sequences

When running massdns in bruteforce mode, the output in JSON will sometimes contain invalid escape sequences (e.g. when using the wordlist on https://gist.github.com/jhaddix/86a06c5dc309d08580a018c66354a056):

~/massdns/scripts/subbrute.py ~/wordlists/all.txt google.com | massdns -r ~/resolvers.txt -t A -o J | grep '\\'

Output:

{"query_name":"\x03\x00\x05\x08\x02\x00\x05\x0d.google.com.","query_type":"A","resp_name":"google.com.","resp_type":"SOA","data":"ns1.google.com. dns-admin.google.com. 241673399 900 900 1800 60"}

This is considered invalid JSON:

echo '{"query_name":"\x03\x00\x05\x08\x02\x00\x05\x0d.google.com.","query_type":"A","resp_name":"google.com.","resp_type":"SOA","data":"ns1.google.com. dns-admin.google.com. 241673399 900 900 1800 60"}' | jq '.'
parse error: Invalid escape at line 1, column 60

A solution could be to encoding as Unicode instead, i.e.

{"query_name":"\u0003\u0000\u0005\u0008\u0002\u0000\u0005\u000d.google.com.","query_type":"A","resp_name":"google.com.","resp_type":"SOA","data":"ns1.google.com. dns-admin.google.com. 241673399 900 900 1800 60"}

Limits can slow massdns?

Hello!
Run as root --root -s 5000, but still uses about 40 mbps (half of the channel). Virtualbox ubuntu 16.04: 64 bit, 12gb ram 3 core, Connection PPPoe.

Maybe problem with limits: ?

root@ri:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 64040
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 100000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 64040
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Cheers,

Add resolver to output

It would be useful to get the resolver that returned the response in the output.
Makes it easier to remove resolvers that are returning bogus responses.

ssh breaks connectivity over massdns

python subbrute.py /lists/names.txt domain.com | massdns -r /lists/resolvers.txt -t A -o S -w ~/massdns_result
curiously, it works on some small domains like domain.com , but most of domains ive been trying, are getting my ssh connection down until the process is over. and guess what ?! blanck output files, im running it on my DigitalOcean server,
would you fellas help me out figuring what i may do to bypass that issue ?
cheers

Inability to output result in a file

When i run massdns -r ~/massdns/lists/resolvers.txt -t A -o S -w massdns.txt alldomains.txt on small list of subdomains, the resolving works fine and i get my output as needed. However, when i run same cmd using commonspeak2-wordlists(~500,000) the result does not ouput to the file? The ouput on terminal is normal but after finishing nothing ouputs to the file. Mark you, some of the inputs from alldomains.txt resolve well when run in a small list of themselves?

The parameter "--type" or "-t" does not seem to work.

The parameter "--type" or "-t" does not seem to work.. From the result, other DNS record types are queried.

My run command:.\bin\massdns.exe -r .\lists\resolvers.txt --type A -o J -w .\bin\qq.json .\bin\qq.txt
image

Is there a way for massdns to query only the specified DNS record type?

Massdns is an excellent tool, looking forward to your reply ~

No buffer space available Error

When I ran massdns in mac OSX EI Capitan, I got the error:

Failed to adjust socket send buffer size.: No buffer space available
Failed to adjust socket receive buffer size.: No buffer space available

How to solve it?

Explicit rr type output

Might not be a bug, but could be by design

If massdns queries for say an MX record and the DNS replies with a CNAME then the a CNAME response is record.

I dont expect to see any record types in the final results that I did not explicit query for, is there an option for that?

Cheers for the awesome project

MassDNS overload Question

Hello,

I have a question.

Where does the massdns tool overload it?

Target site?

Or ISP DNS server?

Thanks.

Feedback & list of issues I am facing

Greetings!!
Thanks for this wonderful tool.
I am facing following issues while using this tool:

  1. The tool is not scanning all domains (when given list of 10000 domains hardly scans 1000 of them which I verified by grep "QUESTION SECTION" in output file generated with -o F switch).
    Although above problem is solved for now by using -s 1000(default is 10000) so I think you should change the default value to 1000.
  2. No explicit option to grep Domain having NXDOMAIN status since it is required for finding domains vulnerable to subdomain takeover(like Azure).
    Although above problem can be solved by :
    ./massdns -q -t A -r /opt/resolvers.txt domain_list.txt -o F -w output_massdns.txt
    cat output_massdns.txt | grep NXDOMAIN -A 7 | grep CNAME
    But it is not efficient.
  3. Getting different result when using -o F and then -w and only using -w (without any output flag)
    Although result should come same.So please tell me if -o F append some extra details which are drop when we are not using it.
  4. Include Option which do below operation:
    Take list of domains and output only those domain names whose IP can be resolved.
    (Should also exclude those domains which have status NO ERROR but only have CNAME and no IP)
    Since this list can be passed further to httprobe(or CURL) or other tools which do some active scanning.(As list fetched above will assure that the domains' IP recored exist).

Explicit "--type" not working

When I run the program with the -t flag, is works perfect but when I try to change it to --type the program breaks with the following error:

Failed to open domain file "--type".

Installation error

HI,

I am getting the error below while compiling the file after cloning from git. Please help.

$ make
mkdir -p bin
cc -O3 -std=c11 -Wall -fstack-protector-strong *.c -o bin/massdns -ldl
In file included from main.c:7:
./massdns.h:7:10: fatal error: 'sys/epoll.h' file not found
#include <sys/epoll.h>
^~~~~~~~~~~~~
1 error generated.
make: *** [all] Error 1

Invalid JSON output produced (but, grabage-in / garbage-out)

Hi @blechschmidt

I was recently performing some testing and didn't realize that my input list had some illegal DNS names in it. Specifically, one of the domains to be queried contained a double quote.

As expected, there was no A record for this name. However, the SOA record was logged and the double quote was not escaped inside the JSON line, causes JSON parsers to choke.

I realize that this was an illegal query, but there should probably be a better way of handling this, either at input time or output time

  1. Reject the name before even querying it and throw a warning to stderr
  2. Escape the name in the JSON output

If you prefer one of these (or none) I am happy to provide a simple PR, please let me know! Thanks

Feature request: time left

Would be nice to have an estimated time left (should be fairly easy since average/second and total is already tracked).

Problem installation with win 10

I cant install with make nolinux commands in windows

Result in cmd :
mkdir -p bin
cc -O3 -std=c11 -Wall -fstack-protector-strong main.c -o bin/massdns
process_begin: CreateProcess(NULL, cc -O3 -std=c11 -Wall -fstack-protector-strong main.c -o bin/massdns, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [Makefile:11: nolinux] Error 2

Benchmark on ZDNS with MassDNS?

ZDNS was another tool to lookup dns records which was developed by ZMap Team. Have you ever do some comparison on ZMap with MassDNS. I wonder which is better for subdomain enumerations.

massdns currently interprets cr/lf in input files as part of the domain name

~ cat resolve.in
example.com

~ file resolve.in
resolve.in: ASCII text

~ cat resolve.in  | massdns -r bin/lists/good_resolvers.txt -t A -s 1
...
;; QUESTION SECTION:
example.com\x0a. IN A

Same behavior happens when using a list of domain names as input parameter, instead of stdin:

~ massdns -s 1 -t A -r bin/lists/good_resolvers.txt resolve.in

;; QUESTION SECTION:
example.com\x0a. IN A

Issue is not present in commit f2457e6, see my comment on this part of code: cafa6ce#r29391735

MassDNS Outputs To Many False Positives

Upon trying to use massdns with the inbuilt subbrut.py module the resulting output contains to many false positives the command which I used was ./scripts/subbrute.py /root/all.txt TARGET.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w OUTPUT.txt and the wordlist I used was all.txt by Jason Haddix the time taken for MassDNS to complete was 3 minutes 01 seconds, I'm using MassDNS on 16.04 on a Digital Ocean Droplet (Singapore Server), any idea how I could fix this?

Stucks when file is empty.

When given domain file is empty, the program stucks and doesnt exit.

My command: massdns -r /tools/massdns/lists/resolvers.txt hosts.txt

wc hosts.txt result: 0 0 0 hosts.txt

Proposal: Option for suppressing the output of specify response types (e.g. NXDOMAIN)

While I realize there is definitely value in logging NXDOMAIN responses, depending on the users' goals, logging NXDOMAIN may be problematic

For example, when surveying a set of a several thousand domains to simply determine the existence of an A record with a brute-force/enumeration list of 100k+ per domain, this results in a ton of disk i/o (and disk space usage) when it may not be useful

Would you consider having a command-line option that would allow the user to specify one or more specific response codes and have logic to sinkhole such responses rather than log them to stdout/stderr/disk? I glanced at the code and I think it's a minor change, essentially just a quick conditional before the fprintf()`, taking a list of which responses types to ignore from the context holding the commandline arguments

I know this can be done by piping the output through grep -v, but it would be nice to be able to specify it when logging the output to a file natively within massdns. It also technically reducing some cycles since fprintf() goes through libc, does some locking and blabla.. so it's a little bit better than filtering on the way out.

I'm happy to have a go at a PR for this if you would accept it. By default, it would filter no records. One or more types could be specified on the command-line as comma separated values, or by using the option more than once, e.g. --ignore NXDOMAIN --ignore FORMERR (probably in most cases you would want to know about FORMERR responses, but this is just an example)

I find a common pattern for my use is as follows:

  1. Generate brute force FQDN lists for each domain I am authorized to enumerate
  2. Run massdns and log to NDJSON format
  3. Select all NOERROR responses via JQ and put the results to a separate file in a specific format for post-processing
  4. Select all SERVFAIL responses via JQ and re-run with these (since often SERVFAIL responses are ephemeral and are "fixed" using multiple passes, probably due to some throttling occurring between the recursive resolver and the authoritative NS that was triggered in the first run)
  5. Repeat 3 and 4 a few times until SERVFAIL totals are constant

In this specific process, NXDOMAIN records are completely ignored and just taking up space- sometimes a significant amount of space. The only use I have for NXDOMAIN in my use-case/process is when testing correctness of recursive resolvers, though I tend to do that outside of massdns anyway

Thoughts?

universal parsing error

13:04:32,063 [ALERT] brute:36 - 探测超时重新探测中
answer = resolver.query(domain, 'A')
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/dns/resolver.py", line 900, in query
timeout = self._compute_timeout(start, lifetime)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/dns/resolver.py", line 799, in _compute_timeout
raise Timeout(timeout=duration)
dns.exception.Timeout: The DNS operation timed out after 60.00314283370972 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "oneforall.py", line 250, in
fire.Fire(OneForAll)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/fire/core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/fire/core.py", line 466, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/fire/core.py", line 675, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "oneforall.py", line 237, in run
self.main()
File "oneforall.py", line 181, in main
brute.run()
File "/Users/elliot/AllFiles/PentestTools/OneForAll/oneforall/brute.py", line 604, in run
valid_subdomains = self.main(self.domain)
File "/Users/elliot/AllFiles/PentestTools/OneForAll/oneforall/brute.py", line 555, in main
self.enable_wildcard = detect_wildcard(domain, ns_ip_list)
File "/Users/elliot/AllFiles/PentestTools/OneForAll/oneforall/brute.py", line 78, in detect_wildcard
wildcard = do_query_a(random_subdomain, resolver)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/tenacity/init.py", line 311, in wrapped_f
return self.call(f, *args, **kw)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/tenacity/init.py", line 391, in call
do = self.iter(retry_state=retry_state)
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/tenacity/init.py", line 350, in iter
raise retry_exc.reraise()
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/tenacity/init.py", line 168, in reraise
raise self.last_attempt.result()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
raise self._exception
File "/Users/elliot/AllFiles/PythonProject/pentest3env/lib/python3.8/site-packages/tenacity/init.py", line 394, in call
result = fn(*args, **kwargs)
File "/Users/elliot/AllFiles/PentestTools/OneForAll/oneforall/brute.py", line 38, in do_query_a
raise tenacity.TryAgain
tenacity.TryAgain

"-o Sm" gives no results

My interpretion of the help suggests that the option "-o Sm" should use simple output and only output records that match the query name. However, no records are output whatsoever. I want to exclude any CNAME records from the output.

2

when I try to brute with a 10MB dictionary, that happens
Empty lookup pool.
Traceback (most recent call last):
File "./scripts/subbrute.py", line 11, in
print(lines.strip() + "." + arg)
IOError: [Errno 32] Broken pipe

and the following situation is not handled
Pan resolve like *.google.xx

Centos 7 Issue

[root@rhynorater massdns]# make
mkdir -p bin
cc  -std=c11 -Wall -fstack-protector-strong *.c -o bin/massdns -lldns -ldl
main.c: In function 'record_from_str':
main.c:129:16: error: 'LDNS_RR_TYPE_CAA' undeclared (first use in this function)
         return LDNS_RR_TYPE_CAA;
                ^
main.c:129:16: note: each undeclared identifier is reported only once for each function it appears in
make: *** [all] Error 1

This is the error I get when I am trying to compile on Centos 7. I have LDNS installed. Any suggestions?

Reading domains from stdin does not give any results

Writing domains to a file and resolve it does give correct results:

# echo www.google.com > domains.txt
# massdns -t A -q -o S -r resolvers.txt domains.txt
www.google.com. A 108.177.14.103
www.google.com. A 108.177.14.106
www.google.com. A 108.177.14.104
www.google.com. A 108.177.14.105
www.google.com. A 108.177.14.99
www.google.com. A 108.177.14.147

Supplying the domains via stdin won't work:

# cat domains.txt | massdns  -t A -q -o S -r resolvers.txt -
In order to use multiprocessing, the domain list needs to be supplied as file.

A few weeks ago this worked. Seems to be a newly introduced bug.

many fake records

Hi,

I was running your tool on my vps , it was running fine for quite sometime but yesterday I installed it on new VPS and I am getting weird result. there is one weird server(142.54.173.92) which is sending fake records , this server does not listed on resolvers.txt(using from your repo). It looks like something is wrong.
I have tried to traceroute this ip but last 10 hopes are not visible and i have also tried to see code massdns but it is in compiled format.

I have copied your repo from my local system to vps wihtout using make command and once i received weird result , i delete entire folder and clone from online using git clone command and use make command .......still i receive this weird records from this weird server ......
It looks like two possible scenario , one of the resolver from list is sending records to this fake dns server(142.54.173.92) which is sending fake records as result.
second could be somewhere in my vps , this records are stored due to using existing copy of the folder and re installation is not removing it .....i grep dns but i could not find any susceptible file

./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -c 100 -w 18_dec_1.txt -

[18_dec_1.txt]

Note : i have changed result to example.com in above command and output file attached.
I really appreciate if you can check on your side that if you are receiving this server in output or something wrong with massdns compiled file or can you provide me any lead to resolve this issue ?

Thanks, Anis

Support DNS over TCP

Hey there @blechschmidt, I generally use massdns over UDP as it makes more sense (much less overhead) but recently I had a case where I was stuck using TCP, due to an unusual connectivity scenario that broke UDP DNS because I was losing the source port- it wasn't a NAT type scenario, it was over a port forward.

I realize that there is a performance penalty with TCP, but it seems like something reasonable to support and I think that while massdns is certainly high-performance, there wouldn't be a severe penalty. Additionally, this would remain as a non-default option

While I would like to write this myself and send a PR-

  1. I'm not sure if you're interested in accepting it
  2. My time is very limited right now (though I understand yours probably is as well)

What are your thoughts on this? Do you agree TCP support would be a reasonable feature to have?

It takes lot of time and -c 200 count does not cover 100000 entry ....

It could be very dump question or some mistake with my configuration of vmware instant ....
I apologies in advance for that ....

It takes lot of time on my machine ....I used it on Kali-Linux-Light-2017.1-vm-amd64 , when I tried domain list of100000 it does not complete with 200 count also ... i used below command ....
./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -w results.txt -
It does not complete even 15000 checks ..... and it takes less than a minute
with below command for 50000 checks ,
./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -c 100 -w results.txt -
It covers 24000 entry in 4-5 mintues .....
my ultimate aim is to use all.txt prepared by Jason(awesome presentation on LevelUp) , I came to know from there about massdns.....

I have allocated 8 cores and 3 gb RAM....It never crosses even 6 digit check.....I even tried with -c 200 count but it takes 14-15 minutes and completes with 75K entry....

Am i missing some configuration of enabling parallel threading ?

Thanks,

[Feature Request] Response Type filterout when logging to file

First of all thanks for this great tool.

I just wanted to ask that is there anyway we can avoid/filterout some response type ("resp_type": "SOA") and only write
other response type to file with NDJSON output?

I know that we can filter out those resp_type with grep -v once the process is done but having a switch --ignore-resp SOA would be great that way we will avoid creating a big file containing few A resp_type and more SOA resp_type.

Thanks.

Mass DNS over HTTP

Hello. I want to tell you about DNS over HTTP. Some people ignores this theme. I want to promote it in terms of mass DNS resolutions.

Let we have an example:

curl -v -H 'accept: application/dns-json' 'https://dns.google.com/resolve?name=fit.com&type=A'

We are not limited to dns.google.com ip addresses. Google allows to proxify dns service via google.*, google.co.*, images.google.com, translate.google.com, etc.

curl -v --resolve dns.google.com:443:173.194.222.138 -H 'accept: application/dns-json' 'https://dns.google.com/resolve?name=fit.com&type=A'

You can "scan" google subnets and find that thousands of ip addresses could proxify DNS.

216.58.192.0/24, 216.58.193.0/24, ...
172.217.18.0/24, 172.217.19.0/24, ...
172.217.22.0/24, 172.217.23.0/24, ...

There are a holes in these subnets (for example 173.194.50.0/24), because dns.google.com cert is not compatible with some google services.

Now let me introduce cloudflare. Their subnets are the best gift ever. It looks like certificate for cloudflare-dns.com is compatible with at least 500 thousands of ip addresses.

curl -v --resolve cloudflare-dns.com:443:104.16.200.100 -H 'accept: application/dns-json' 'https://cloudflare-dns.com/dns-query?name=fit.com&type=A'

104.16.0.0/16, 104.17.0.0/16, 104.18.0.0/16, ...

It is possible to accumulate several millions of endpoints and provide good load on google and cloudflare.


Disadvantages:

  1. Google and Cloudflare could ban client ip addresses if it will provide huge load on their network.
  2. DNS over HTTP is provided by these companies only.

PS Do not forget that we could process multiple DNS queries via single tcp connection before connection will become broken. "keep-alive" or HTTP/2 will improve this story.

Thank you.

massdns installation error on ubuntu

root@ubuntu:~/tool/massdns# make

mkdir -p bin
cc  -O3 -std=c11 -DHAVE_EPOLL -DHAVE_SYSINFO -Wall -fstack-protector-strong src/main.c -o bin/massdns
cc: error: unrecognized command line option ‘-fstack-protector-strong’
make: *** [all] Error 1

I am using Ubuntu 14.04.6 LTS

About the resolvers.txt

I wanted to know if a big amount of resolvers is needed for the tool to work properly. i tried running the tool with only 10 public and common DNS resolvers and the tool had less "REFUSED", does that mean that is running ok?

What would be a good value for -s if i have only 10 resolvers in resolvers.txt?

accept from stdin not working

The example on the homepage using subbrute: $ ./scripts/subbrute.py lists/names.txt example.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w results.txt does not work.

It appears massdns requires to be fed an input file and I do not see any way to force it to accept domains from stdin. any suggestions?

Publish image in Docker Hub

I suggest automatic deployment of the Dockerfile in Docker Hub.

This is a more easier way of start to use the tool without to clone the repo and perform manual steps.

Integration in Docker Hub is as faster as link both repos so Docker Hub starts to build new images as branch pushes occurs

https://docs.docker.com/docker-hub/builds/

Problem with multiple filter option.

Hi !
i am seeing that filter option does not return any result for some states, by other hand it's not possible to choose multiple filters.
For example, how can filter NXDOMAIN,SERVFAIL and REFUSED only ignoring OK or NOERROR status and dump them into a file?. What i am missing?

Thanks in advance.

Sometimes massdns doesn't seem to work properly.

My test domain is guahao.com

Authoritative DNS nameservers of guahao.com in is: ns3.dnsv4.com and ns4.dnsv4.com

root@kali:~/Desktop/massdns# dig guahao.com ns

; <<>> DiG 9.11.5-P4-3-Debian <<>> guahao.com ns
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6183
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;guahao.com.			IN	NS

;; ANSWER SECTION:
guahao.com.		5	IN	NS	ns3.dnsv4.com.
guahao.com.		5	IN	NS	ns4.dnsv4.com.

;; Query time: 469 msec
;; SERVER: 192.168.197.2#53(192.168.197.2)
;; WHEN: Sat Mar 28 14:39:24 EDT 2020
;; MSG SIZE  rcvd: 70

The A record of the authoritative DNS name server is

162.14.18.121
129.211.176.242
183.232.90.141
52.74.43.18
14.215.150.14
61.151.180.49
59.36.120.145
61.129.8.140
52.74.43.18
14.215.150.15
111.30.142.18
223.166.151.14
125.39.45.245
61.151.180.50
223.166.151.15
162.14.18.188
58.251.86.11

I can use dig to query any subdomain under guahao.com

root@kali:~/Desktop/massdns# dig bbs.guahao.com @14.215.150.15

; <<>> DiG 9.11.5-P4-3-Debian <<>> bbs.guahao.com @14.215.150.15
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30045
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 453918193b536bf0 (echoed)
;; QUESTION SECTION:
;bbs.guahao.com.			IN	A

;; ANSWER SECTION:
bbs.guahao.com.		600	IN	CNAME	bbs.guahao.com.cdn.dnsv1.com.

;; AUTHORITY SECTION:
guahao.com.		86400	IN	NS	ns4.dnsv4.com.
guahao.com.		86400	IN	NS	ns3.dnsv4.com.

;; Query time: 44 msec
;; SERVER: 14.215.150.15#53(14.215.150.15)
;; WHEN: Sat Mar 28 14:47:26 EDT 2020
;; MSG SIZE  rcvd: 151

But massdns always has no results, and there are no results until the end of the run.

Processed queries: 676
Received packets: 0
Progress: 100.00% (00 h 00 min 26 sec / 00 h 00 min 26 sec)
Current incoming rate: 0 pps, average: 0 pps
Current success rate: 0 pps, average: 0 pps
Finished total: 676, success: 0 (0.00%)
Mismatched domains: 0 (0.00%), IDs: 0 (0.00%)
Failures: 0: 0.00%, 1: 0.00%, 2: 0.00%, 3: 0.00%, 4: 0.00%, 5: 0.00%, 6: 0.00%, 7: 0.00%, 8: 0.00%, 9: 0.00%, 10: 0.00%, 11: 0.00%, 12: 0.00%, 13: 0.00%, 14: 0.00%, 15: 0.00%, 16: 0.00%, 17: 0.00%, 18: 0.00%, 19: 0.00%, 20: 0.00%, 21: 0.00%, 22: 0.00%, 23: 0.00%, 24: 0.00%, 25: 0.00%, 26: 0.00%, 27: 0.00%, 28: 0.00%, 29: 0.00%, 30: 0.00%, 31: 0.00%, 32: 0.00%, 33: 0.00%, 34: 0.00%, 35: 0.00%, 36: 0.00%, 37: 0.00%, 38: 0.00%, 39: 0.00%, 40: 0.00%, 41: 0.00%, 42: 0.00%, 43: 0.00%, 44: 0.00%, 45: 0.00%, 46: 0.00%, 47: 0.00%, 48: 0.00%, 49: 0.00%, 50: 100.00%, 
Response: | Success:               | Total:
OK:       |            0 (  0.00%) |            0 (  0.00%)
NXDOMAIN: |            0 (  0.00%) |            0 (  0.00%)
SERVFAIL: |            0 (  0.00%) |            0 (  0.00%)
REFUSED:  |            0 (  0.00%) |            0 (  0.00%)
FORMERR:  |            0 (  0.00%) |            0 (  0.00%)

My command is
D:\OneForAll\oneforall\thirdparty\massdns\windows\x64\massdns_windows_amd64.exe --status-format ansi --processes 1 --socket-count 1 --resolvers D:\OneForAll\oneforall\data\authoritative_dns.txt --resolve-count 50 --type A --flush --output J --outfile D:\OneForAll\oneforall\thirdparty\massdns\temp\resolved_result_guahao.com_20200329_024745.json --error-log D:\OneForAll\oneforall\thirdparty\massdns\temp\massdns_error_log.txt D:\OneForAll\oneforall\thirdparty\massdns\temp\generated_subdomains_guahao.com_20200329_024745.txt
authoritative_dns.txt
generated_subdomains_guahao.com_20200329_024745.txt

Looking forward to your reply very much, thank you!😊

Multiple source ports

Is it possible to have multiple ports rather than re-using same port...if one dns request is detected by threat as a firewall, all consequent queries match same session and gets discarded

wildcard subdomains

I get a list of subdomains with other scripts.
After that I get DNS records and based on this I get valid subdomains.
But often I get a lot of false results, in particular because of the fact that the subdomain has a wildcard configured on the domain.
How to enable the function of checking the wildcard subdomains so that I get cleaner results that exclude the wildcard subdomains?
I see massdns have this function "Add wildcard detection for reconnaissance"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.