1an0rmus / tekdefense-automater Goto Github PK

Changing the fortinet section to reflect as below fixed the issue I was having where it would show No results found:

<site name="fortinet_classify">
    <sitetype>
        <entry>ip</entry>
        <entry>hostname</entry>
    </sitetype>
    <domainurl>https://fortiguard.com/webfilter</domainurl>       (added webfilter)
    <reportstringforresult>
        <entry>[+] Fortinet URL Category:</entry>
    </reportstringforresult>
    <sitefriendlyname>
        <entry>FNet URL</entry>
    </sitefriendlyname>
    <regex>
        <entry>Category:\s(.+)\&lt;\/h3\&gt;\s\&lt;a</entry>
    </regex>
    <fullurl>https://www.fortiguard.com/iprep?data=%TARGET%</fullurl>         (url now uses the current webfilter page)
    <importantproperty>
        <entry>Results</entry>
    </importantproperty>
</site>

Hello -

Trying to work through the same issue that was reported and closed back in November 24, 2015. Original post from user is below :

python Automater.py 4.4.4.4
Traceback (most recent call last):
File "Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/home/yo/other/TekDefense-Automater/siteinfo.py", line 40, in
requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'

This user updated urllib3 and requests. I have this exact same issue on fresh installs of Ubuntu 14.0.4 and 16.0.4. I did troubleshooting through python, etc...

The install method i followed was from this site.

Thanks for any help...

First off, endless thanks for this very cool tool. I'm having a slight issue (I imagine it's either library or syntax related). Whenever I have Automater scan a .txt file containing IP's it returns that none of them are blacklisted by IPVoid (although I have verified that some of them are blacklisted by manually entering in the website).

'Python Automater.py 131.221.84.34' returns IPVoid blacklist of "No results found" but when manually entered in the IPVoid website it returns a score of 1/83. Attempted running with Python 2.7 but same results. I've attempted the -post option but it looks like that is no longer utilized in the most recent version. -Vv returns that "All Automater files are up to date". Changed Tekdefense.xml to point to http://www.ipvoid.com/ip-blacklist-check/ but it still returns "No results found". MalC0de and ProjectHoneypot also return no results found. Thanks for your help!

For example for pl400.dyndns.org, the script does not report DNS-BH as being present, while it is when checking the site: http://www.urlvoid.com/scan/pl400.dyndns.org

First of all, thank you for making this tool, it is a great tiemsaver.
I found a small limitation that may be already well known, but I feel it makes the tool a little less convenient for users.

First I have cloned the git repo to /opt/TekDefense-Automater.
If I run it from inside that directory, everything works:

$ python Automater.py google.com
[*] Checking http://www.fortiguard.com/ip_rep/index.php?data=google.com&lookup=Lookup
...

However, if I run it from outside that directory, for example in /tmp, the script doesn't output anything. No results, no errors.

$ cd /tmp
$ python /opt/TekDefense-Automater/Automater.py google.com
$

It would be nice if you could call the script from anywhere, for example with an alias.

This is not really an issue but only a wrong default value. I have noticed that the default value in one of the regex for virustotal is: "},\s"(.{3,20})"....detected...true.{19,32}result....(.{11,30})....update"
I have found that the pair {11,30} is not valid for some results, i.e.:
md5: f1e090f4d1f10a7abe586d49012dbd52
[+] Detected Engines: 1
[+] Vendor | Classification: No results found
Here the problem is that the max. value 30 is too short in this case because the string is 33 characters long: "PE:Malware.Generic/QRS!1.9E2D [F]".
I would propose to change that max. value to 40 to avoid similar exceptional strings.
Thank you for this great program!

Hi,

I have downloaded the latest files and placed all of them in the same folder. I am using a Windows 7 machine with Python 2.7.

When i run the Automater.py, the script runs quietly but the output just says, "No results found".

Could you please help me here.

Thanks,
Cooldude

Running Ubuntu 14.10

Trace

Traceback (most recent call last):
File "Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/home/seifreed/tools/malware/TekDefense-Automater/siteinfo.py", line 39, in
requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'

Ist this project still under development

Lately ipvoid have changed their website layout and now the POST option for unscanned IP addresses doesn't work. I could find the time to fix that by myself (the first step would be to correct the regex in sites.xml) but I might need some guidance.

http://www.reputationauthority.org/lookup.php?ip=194.25.152.9

To disable the self-signed certificate warning, change line 39 in siteinfo.py from

requests.packages.urllib3.disable_warnings

to

from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

Hi. I'm new here and to linux I am having a bit of trouble getting Automater set up. I've done my best to search around but can't find an answer to my issue. This is the error I receive when trying to run Automater:

Traceback (most recent call last):
File "/usr/local/bin/Automater.py", line 40, in
from siteinfo import SiteFacade, Site
File "/usr/local/bin/siteinfo.py", line 35, in
from outputs import SiteDetailOutput
ImportError: No module named outputs

Any help would be greatly appreciated.

Thanks!

The regular expressions used to identify IP addresses and IP addresses with a netmask (as seen in IPWrapper.isIPorIPList and IPWrapper.getTarget) are too accepting of input related to parsing numbers. The regular expression is defined as:
'\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}/\d{1,2}'
An example input that shows this error is: 999.999.999.999/99

The option re.IGNORECASE is provided as to show case insensitivity. However, this option does nothing as no letters are contained within the expression.
Additionally, the regular expression is not bounded, matching content that has the appearance of an IP address within other content.

An example input that shows this error is: 1.1.1.1/24AAA

In order to validate IPv4 addresses with a netmask in the canonical form, the following regular expression would be more valid:
'^(?:(?:\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]).){3}(?:\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])/(?:[1-9]|[1-2]\d|3[0-2])$'

Recently noticed that the tool is no longer pulling down VT data. I've checked the page manually against the regex, they haven't changed anything there and the URL's still valid so I think it must not be getting the data properly. Not sure how to debug that!

Updating the delay option in utilities.py from int to float would provide the user with a little more control.

Original: self._parser.add_argument('-d', '--delay', type=int, default=2, help='This will change the delay to the inputted seconds. Default is 2.')

Update: self._parser.add_argument('-d', '--delay', type=float, default=2, help='This will change the delay to the inputted seconds. Default is 2.')

Thank you,
cutaway

Hello,
For a possible next release it would be great take a tagged release in git.
In kali we have tools that monitors web pages listing release, and it works well with github pages showing git tags. If you make a tagged release we will be automatically informed and we will update the package quickly.
Thanks

https://threatrecon.co

Will need to force the user to add a free API key.
Be ware there is a rate limiting of something like 4/minute

http://www.blacklistalert.org/

With respect to the inclusion of more sites: https://isc.sans.edu/api/#ip - would be marvellous to have this. I see you have expressed the intent to write a guide on how to play around with Sites.xml - I'm a fast learner so even if you had something very sketchy it would still give me a good starting point. Much appreciating what you've done already :)

[update] pretty sure I've figured out how to include this myself - it's a good motivator to start learning to use XML anyway.

Getting this error after trying run the latest code push from the git repo

yo@supfoo ~/other/TekDefense-Automater $ python Automater.py 4.4.4.4
Traceback (most recent call last):
  File "Automater.py", line 40, in <module>
    from siteinfo import SiteFacade, Site
  File "/home/yo/other/TekDefense-Automater/siteinfo.py", line 40, in <module>
    requests.packages.urllib3.disable_warnings()
AttributeError: 'module' object has no attribute 'packages'
yo@supfoo ~/other/TekDefense-Automater $

Fortinet changed their classification page. The following regex will fix it

\<h1\>https?:\/\/(.+):\s(.+)\<\/h1\>

Using the regex below worked for totalhash_md5

/analysis/?(\w{40}).+

Also include detection rates for latest detected URLs and hashes

Greetings,

One site where I work has very little asset tracking ability. A quick win for them is doing a DNS PTR lookup on an IP and seeing what it resolves to for local IPs.

Could you add a feature that would do this?

If I find the time this weekend, I'll try to get it done myself.