1an0rmus / tekdefense-automater Goto Github PK
View Code? Open in Web Editor NEWAutomater - IP URL and MD5 OSINT Analysis
Home Page: www.tekdefense.com
License: MIT License
Automater - IP URL and MD5 OSINT Analysis
Home Page: www.tekdefense.com
License: MIT License
This is not really an issue but only a wrong default value. I have noticed that the default value in one of the regex for virustotal is: "},\s"(.{3,20})"....detected...true.{19,32}result....(.{11,30})....update"
I have found that the pair {11,30} is not valid for some results, i.e.:
md5: f1e090f4d1f10a7abe586d49012dbd52
[+] Detected Engines: 1
[+] Vendor | Classification: No results found
Here the problem is that the max. value 30 is too short in this case because the string is 33 characters long: "PE:Malware.Generic/QRS!1.9E2D [F]".
I would propose to change that max. value to 40 to avoid similar exceptional strings.
Thank you for this great program!
Also include detection rates for latest detected URLs and hashes
To disable the self-signed certificate warning, change line 39 in siteinfo.py from
requests.packages.urllib3.disable_warnings
to
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
Getting this error after trying run the latest code push from the git repo
yo@supfoo ~/other/TekDefense-Automater $ python Automater.py 4.4.4.4
Traceback (most recent call last):
File "Automater.py", line 40, in <module>
from siteinfo import SiteFacade, Site
File "/home/yo/other/TekDefense-Automater/siteinfo.py", line 40, in <module>
requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'
yo@supfoo ~/other/TekDefense-Automater $
Hi. I'm new here and to linux I am having a bit of trouble getting Automater set up. I've done my best to search around but can't find an answer to my issue. This is the error I receive when trying to run Automater:
Traceback (most recent call last):
File "/usr/local/bin/Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/usr/local/bin/siteinfo.py", line 35, in
from outputs import SiteDetailOutput
ImportError: No module named outputs
Any help would be greatly appreciated.
Thanks!
First of all, thank you for making this tool, it is a great tiemsaver.
I found a small limitation that may be already well known, but I feel it makes the tool a little less convenient for users.
First I have cloned the git repo to /opt/TekDefense-Automater.
If I run it from inside that directory, everything works:
$ python Automater.py google.com
[*] Checking http://www.fortiguard.com/ip_rep/index.php?data=google.com&lookup=Lookup
...
However, if I run it from outside that directory, for example in /tmp, the script doesn't output anything. No results, no errors.
$ cd /tmp
$ python /opt/TekDefense-Automater/Automater.py google.com
$
It would be nice if you could call the script from anywhere, for example with an alias.
Using the regex below worked for totalhash_md5
/analysis/?(\w{40}).+
Running Ubuntu 14.10
Trace
Traceback (most recent call last):
File "Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/home/seifreed/tools/malware/TekDefense-Automater/siteinfo.py", line 39, in
requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'
Greetings,
One site where I work has very little asset tracking ability. A quick win for them is doing a DNS PTR lookup on an IP and seeing what it resolves to for local IPs.
Could you add a feature that would do this?
If I find the time this weekend, I'll try to get it done myself.
With respect to the inclusion of more sites: https://isc.sans.edu/api/#ip - would be marvellous to have this. I see you have expressed the intent to write a guide on how to play around with Sites.xml - I'm a fast learner so even if you had something very sketchy it would still give me a good starting point. Much appreciating what you've done already :)
[update] pretty sure I've figured out how to include this myself - it's a good motivator to start learning to use XML anyway.
Recently noticed that the tool is no longer pulling down VT data. I've checked the page manually against the regex, they haven't changed anything there and the URL's still valid so I think it must not be getting the data properly. Not sure how to debug that!
Lately ipvoid have changed their website layout and now the POST option for unscanned IP addresses doesn't work. I could find the time to fix that by myself (the first step would be to correct the regex in sites.xml) but I might need some guidance.
First off, endless thanks for this very cool tool. I'm having a slight issue (I imagine it's either library or syntax related). Whenever I have Automater scan a .txt file containing IP's it returns that none of them are blacklisted by IPVoid (although I have verified that some of them are blacklisted by manually entering in the website).
'Python Automater.py 131.221.84.34' returns IPVoid blacklist of "No results found" but when manually entered in the IPVoid website it returns a score of 1/83. Attempted running with Python 2.7 but same results. I've attempted the -post option but it looks like that is no longer utilized in the most recent version. -Vv returns that "All Automater files are up to date". Changed Tekdefense.xml to point to http://www.ipvoid.com/ip-blacklist-check/ but it still returns "No results found". MalC0de and ProjectHoneypot also return no results found. Thanks for your help!
For example for pl400.dyndns.org, the script does not report DNS-BH as being present, while it is when checking the site: http://www.urlvoid.com/scan/pl400.dyndns.org
Fortinet changed their classification page. The following regex will fix it
\<h1\>https?:\/\/(.+):\s(.+)\<\/h1\>
Will need to force the user to add a free API key.
Be ware there is a rate limiting of something like 4/minute
The regular expressions used to identify IP addresses and IP addresses with a netmask (as seen in IPWrapper.isIPorIPList and IPWrapper.getTarget) are too accepting of input related to parsing numbers. The regular expression is defined as:
'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}/\d{1,2}'
An example input that shows this error is: 999.999.999.999/99
The option re.IGNORECASE is provided as to show case insensitivity. However, this option does nothing as no letters are contained within the expression.
Additionally, the regular expression is not bounded, matching content that has the appearance of an IP address within other content.
An example input that shows this error is: 1.1.1.1/24AAA
In order to validate IPv4 addresses with a netmask in the canonical form, the following regular expression would be more valid:
'^(?:(?:\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]).){3}(?:\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])/(?:[1-9]|[1-2]\d|3[0-2])$'
Ist this project still under development
Updating the delay option in utilities.py from int
to float
would provide the user with a little more control.
Original: self._parser.add_argument('-d', '--delay', type=int, default=2, help='This will change the delay to the inputted seconds. Default is 2.')
Update: self._parser.add_argument('-d', '--delay', type=float, default=2, help='This will change the delay to the inputted seconds. Default is 2.')
Thank you,
cutaway
Hello -
Trying to work through the same issue that was reported and closed back in November 24, 2015. Original post from user is below :
python Automater.py 4.4.4.4
Traceback (most recent call last):
File "Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/home/yo/other/TekDefense-Automater/siteinfo.py", line 40, in
requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'
This user updated urllib3 and requests. I have this exact same issue on fresh installs of Ubuntu 14.0.4 and 16.0.4. I did troubleshooting through python, etc...
The install method i followed was from this site.
Thanks for any help...
Hello,
For a possible next release it would be great take a tagged release in git.
In kali we have tools that monitors web pages listing release, and it works well with github pages showing git tags. If you make a tagged release we will be automatically informed and we will update the package quickly.
Thanks
Changing the fortinet section to reflect as below fixed the issue I was having where it would show No results found:
<site name="fortinet_classify">
<sitetype>
<entry>ip</entry>
<entry>hostname</entry>
</sitetype>
<domainurl>https://fortiguard.com/webfilter</domainurl> (added webfilter)
<reportstringforresult>
<entry>[+] Fortinet URL Category:</entry>
</reportstringforresult>
<sitefriendlyname>
<entry>FNet URL</entry>
</sitefriendlyname>
<regex>
<entry>Category:\s(.+)\<\/h3\>\s\<a</entry>
</regex>
<fullurl>https://www.fortiguard.com/iprep?data=%TARGET%</fullurl> (url now uses the current webfilter page)
<importantproperty>
<entry>Results</entry>
</importantproperty>
</site>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.