opsdisk / metagoofil Goto Github PK

View Code? Open in Web Editor NEW

388.0 21.0 80.0 67 KB

Search Google and download specific file types

License: Other

Python 98.53% Dockerfile 1.47%

python osint metagoofil google dork information-leakage penetration-testing

metagoofil's People

Contributors

Stargazers

Watchers

Forkers

cerebralmischief phpplay blkstone techwizmac fo0nikens mlinton cloudsecuritylabs illordlo natrix-fork zerof gavz yorkeyanyy 0xsanjaydas umbrell4 kyzenator 3v1lw1th1n johndoehack khr0x40sh 5l1v3r1 skidave preventions dkitdfir lexus89 hacking-repo millsapcyber arpsabbir asyoungeio superkojiman cknabs jeffbarron argonauta666 sashka3076 mathevsval dhavalu585 baifengbai kboghdady jrgdiaz dbaruajuniv naeiiim blackhatethicalhacking fixgroup21 jhoan1 mobilelifeful excloudx6 dormeno lowgun piotrm3x6 jgoodacre93 willbui69 morcelli himalaya-pamir camden36 tracesrc gabbelebab developers81828182 atdavidpark yen5004 zerochaos- cyberraya7 gongolik shashankhacker730 scpanddeveloping mrmoshkovitz thomas-ayissi ethicalsecurity-agency kptcheesewhiz brentguttmann sixyuandata janedoe667 ethicalsecurity-agency lcsouzamenezes fillipino2 suleymansonmezcan gairxx asuenaga819 eduardompc rifcheesy 6un9-h0-dan

metagoofil's Issues

Float and int error

Both of your functions for positive_int and positive_float are broken. I think the issue is that you are calling type on the value which can't be greater than 0 since type is not an integer or float.

Google similar entries

Hi,

When doing a search and google does not show all files but instead he show 1 or a few and then there is the following message:

In order to show you the most relevant results, we have omitted some entries very similar to the 1 already displayed.
If you like, you can repeat the search with the omitted results included.

The program only see the first item found by Google and will stop there. It is a problem as I've got some example where 4 to 5 pages of results does offers PDF but only one is shown.

Let me know if you know a way to fix it or if you need help for the code/test.

Thanks

No module named google

Having installed pip install google, this version of metagoofil errors with "ImportError: No module named google"

My fix was to replace "google" by "googlesearch" 3 times in lines 7 and 105 of metagoofil.py.

Add User-Agent option?

Might it be possible to add a user-agent impersonation option for accessing sites? I've run across things like this a few times: https://stackoverflow.com/questions/25936072/python-urllib2-httperror-http-error-503-service-unavailable-on-valid-website and having a way to set a user agent with a flag would be phenomenal

Can we make this compatible with other operating systems such as macOS, or Windows? I am a really big fan of this tool, and would love to see its availability globalized. I have tried running it on my macOS system, and, no matter how much I modify the code, I cannot get it to work. I figured I would reach out to the devs! Huge props, by the way.

Fix for one output error

Hi,

Was getting this error:

Traceback (most recent call last):
File "./metagoofil.py", line 226, in
mg.go()
File "./metagoofil.py", line 146, in go
self.html_links.close()
AttributeError: Metagoofil instance has no attribute 'html_links'

Fix

From:

            if self.save_links:
                for f in self.files:
                    self.html_links.write(f + "\n")

        self.html_links.close()

To:


            if self.save_links:
                for f in self.files:
                    self.html_links.write(f + "\n")
        if self.save_links:
            self.html_links.close()

urllib2.HTTPError: HTTP Error 503: Service Unavailable

Can't pinpoint what is causing this, it usually happens after a big search (-l 500 for example), but after that if persists to even the smallest searches (as seen below)

after that it keeps giving that error no matter what i do and eventually it works for one more run.

# python metagoofil.py -d apple.com -t pdf,doc -l 50 -n 5 -o test -w -f
[*] Downloaded files will be saved here: test
[*] Searching for 50 .pdf files and waiting 30.0 seconds between searches
Traceback (most recent call last):
  File "metagoofil.py", line 226, in <module>
    mg.go()
  File "metagoofil.py", line 124, in go
    for url in googlesearch.search(query, start=0, stop=self.search_max, num=100, pause=self.delay, extra_params={'filter': '0'}, user_agent=self.user_agent):
  File "/usr/local/lib/python2.7/dist-packages/googlesearch/__init__.py", line 359, in search
    html = get_page(url)
  File "/usr/local/lib/python2.7/dist-packages/googlesearch/__init__.py", line 147, in get_page
    response = urlopen(request)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 503: Service Unavailable

Any ideas?

to clarify: the fork is missing all modules

It seems like you are trying to continue the original work and this would be really cool. However, your "fork" has much lesser code. it's 550kb (old) vs 15kb (new).

Could you clarify it please?

`pip install -r requirements.txt` not working, results in `No matching distribution found for requests==2.28.1`

I do pip install -r requirements.txt
This results in

 pip install -r requirements.txt
Collecting google==3.0.0
  Using cached google-3.0.0-py2.py3-none-any.whl (45 kB)
ERROR: Could not find a version that satisfies the requirement requests==2.28.1 (from versions: 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.6.4, 0.6.5, 0.6.6, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5, 0.8.6, 0.8.7, 0.8.8, 0.8.9, 0.9.0, 0.9.1, 0.9.2, 0.9.3, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.4, 0.10.6, 0.10.7, 0.10.8, 0.11.1, 0.11.2, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.13.4, 0.13.5, 0.13.6, 0.13.7, 0.13.8, 0.13.9, 0.14.0, 0.14.1, 0.14.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.1, 2.3.0, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0, 2.6.1, 2.6.2, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.11.1, 2.12.0, 2.12.1, 2.12.2, 2.12.3, 2.12.4, 2.12.5, 2.13.0, 2.14.0, 2.14.1, 2.14.2, 2.15.1, 2.16.0, 2.16.1, 2.16.2, 2.16.3, 2.16.4, 2.16.5, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0, 2.18.1, 2.18.2, 2.18.3, 2.18.4, 2.19.0, 2.19.1, 2.20.0, 2.20.1, 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0, 2.25.1, 2.26.0, 2.27.0, 2.27.1)
ERROR: No matching distribution found for requests==2.28.1

doesnt play nicely when it receives a status 429

Hello,
When you run this command(in kali):
metagoofil -d https://sans.org -t doc,pdf,xls -l 200 -o sans_files -f
It fails instead of correctly handling this exception:
urllib.error.HTTPError: HTTP Error 429: Too Many Requests

Issue was first reported here:
https://bugs.kali.org/view.php?id=7005

python3.10 Running Error

I encountered this problem after running.

$ proxychains -q python3 metagoofil.py -d nsfocus.com -f -t all -e 60 
[*] Searching for 100 .all files and waiting 60.0 seconds between searches
[-] EXCEPTION: <urlopen error EOF occurred in violation of protocol (_ssl.c:997)>
Traceback (most recent call last):
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/local/python3/lib/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/usr/local/python3/lib/python3.10/http/client.py", line 1454, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/local/python3/lib/python3.10/ssl.py", line 512, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/python3/lib/python3.10/ssl.py", line 1070, in _create
    self.do_handshake()
  File "/usr/local/python3/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 162, in go
    for url in googlesearch.search(
  File "/usr/local/python3/lib/python3.10/site-packages/googlesearch/__init__.py", line 272, in search
    get_page(url_home % vars(), user_agent, verify_ssl)
  File "/usr/local/python3/lib/python3.10/site-packages/googlesearch/__init__.py", line 174, in get_page
    response = urlopen(request)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/local/python3/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:997)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 381, in <module>
    mg.go()
  File "/home/osint/osint/domain_col/tool/metagoofil/metagoofil.py", line 175, in go
    if e.code == 429:
AttributeError: 'URLError' object has no attribute 'code'

opsdisk / metagoofil Goto Github PK

metagoofil's People

Contributors

Stargazers

Watchers

Forkers

metagoofil's Issues

Float and int error

Google similar entries

No module named google

Add User-Agent option?

Feature suggestion

Fix for one output error

urllib2.HTTPError: HTTP Error 503: Service Unavailable

to clarify: the fork is missing all modules

`pip install -r requirements.txt` not working, results in `No matching distribution found for requests==2.28.1`

doesnt play nicely when it receives a status 429

python3.10 Running Error

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent