lockfale / osint-framework Goto Github PK
View Code? Open in Web Editor NEWOSINT Framework
License: MIT License
OSINT Framework
License: MIT License
First off..awesome work doing this. One of the simplest ways I've come across to organise a mass amount of resources into logical groupings. So I'm a noob to JSON and SVG, but having a look at the code in arf.js and chatting with mr google:
nodeEnter.append("svg:title")
.text(function(d) {
return d.tip;
});
Then add a "tip" field to the JSON child item (in fact, adding "tip" to any item adds a mouseover tip) :
"id": "502",
"name": "Namechk",
"type": "url",
"url": "https://namechk.com/",
"tip": "This is the tool tip for Namechk\nThis is line 2 of my tip"
On mouseover, your tooltip pops up, giving additional info as written in the JSON source.
Just a thought?
Cheers.
JN,
Just curious what you are generating your arf.json from. are you converting from .mm or some other standard? interested in sharing the script, if so?
doesn't hurt to ask I hope :)
Thanks,
Link to a Keyword Analysis under Domain Names -> Analytics leads to a Page not found page.
Hi, thanks for a great framework.
I noticed, that in the list there are many dorks (google queries) and from the first view they are undistinguishable from all the other links. Just as enhancement I can propose marking the dorks with (d) as you currently do with tools (t)
Wish I had time to do a PR but I'll post this pretty good tool here for later inclusion in the framework: https://github.com/m4ll0k/infoga
These tools are no longer available:
Social Networks -> Twitter ->
http://nearbytweets.com/
https://www.allmytweets.net/connect.php
Hi, I'm the author of Belati( https://github.com/aancw/Belati )
Belati - The Traditional Swiss Army Knife For OSINT
Belati is tool for Collecting Public Data & Public Document from Website and other service for OSINT purpose. This tools is inspired by Foca and Datasploit for OSINT :)
I want to make pull request, but i don't know what number should i use for id. 195 is already used by folder:
{
"id": "195",
"name": "Belati (T)",
"type": "url",
"url": "https://github.com/aancw/Belati"
}],
Thanks!
Hello,
Recently I released a utility written in Bash designed to download and install a number of high quality OSINT, Threat Intel and Recon tools. Not only does it perform an installation of these tools it also manages the installation of any dependencies required by them . Most of the tools available in my implementation are already featured in OSINT-Framework. The aim of my shell script however, is to automate the deployment of the tools it features, and to make setting up an intelligence gathering environment quick and easy.
Here is a quick summary of the programs available with IntRec-Pack
+-----------------------+-------------------------------------------+
| Tool | Utility type and feature summary |
+-----------------------+-------------------------------------------+
|1. QuickScan | Port Scanner/WHOIS/Domain Resolver |
|2. DNSRecon | Advanced DNS Enumeration & Domain Utility |
|3. Sublist3r | OSINT Based Subdomain Enumeration |
|4. TekDefense-Automator| OSINT Based IP, URL and Hash Analyzer |
|5. TheHarvester | eMail, vHost, Domain and PII Enumeration |
|6. IOC-Parser | Threat Intel, parses IOC data from reports|
|7. PyParser-CVE | Multi Source Exploit Parser/CVE Lookup |
|8. Mimir | HoneyDB CLI/Threat Intelligence Utility |
|9. Harbinger | Cymon.io, Virus Total, Threat Feed Parser |
|10.Spiderfoot | Advanced OSINT/Reconnaissance Framework |
+-----------------------+-------------------------------------------+
Furthermore I have added a feature that will allow the user to visit http://osintframework.com by opening it programatically from within the Bash script. The reason for which being that if the user should find they are in need of an OSINT tool that is not covered by IntRec-Pack they can easily browse the selection of resources that you are providing in order to complete their own arsenal.
Should you like to review the source code and/or details of the project please visit it's official repo by clicking here.
With kind regards,
NullArray
Hey there,
It seems the link is broken.
Best,
WhatWeb - Web application and web server fingerprinting tool.
Domain Name -> Analytics:
{
"id": "<ID>",
"name": "WhatWeb (T)",
"type": "url",
"url": "https://github.com/urbanadventurer/WhatWeb"
},
http://www.newsobserver.com/news/databases/state-pay/
A bit specific to NC but lots of good info here. You can search by name, etc.
So the sites in the Username -> Specific sites section...what are the thoughts about including my https://github.com/WebBreacher/WhatsMyName project in there? The project has over 160 sites with user name enumeration but it is mostly used in Recon-ng, Spiderfoot and using the script in the project.
Do we put every site where you can do user name enumeration or add those to a list/project and note the project in that section?
I would have made a PR, but readme.md doesn't explain how "arf.json" is set, so i don't know the latest "id" and how to set the link's category.
URL: https://www.leakedsource.com/
Section: Email Address -> Breach data
Would you consider adding https://haveibeenpwned.com to the Adv Recon Framework -> Email Address -> Breach Data -> haveibeenpwned.com?
Hi again
What is a good editor for json files?
I find it really hard to edit this file with notepad++
Thanks !!
Search Engines -> Code Search -> Meanpath
meanpath.com’s server DNS address could not be found
Not sure whether it's my DNS problem or a global one
Search Engines -> Search Tools
Google Alerts are listed twice
Email Address -> Email Search -> Find Any Email Checker
default backend - 404 error
I'm installing this in a Lubuntu VM I use for forensics. I am not a regular user of node/npm/bower.
First, it wasn't clear that node needed to be installed, maybe say more about prerequisites for someone starting from scratch?
Then this happened:
./node_modules/bower/bin/bower install
bower invalid-meta for:/home/hunch/OSINT-Framework/bower.json
bower invalid-meta The "name" is recommended to be lowercase, can contain digits, dots, dashes
https://github.com/lockfale/ARF redirects to https://github.com/lockfale/OSINT-Framework, not sure if that's a problem, I manually fixed it in bower.json. After this I read bower help, check, and this is installed.
./node_modules/bower/bin/bower list
bower check-new Checking for new versions of the project dependencies...
arf /home/hunch/OSINT-Framework
└── d3#3.5.16 (latest is 4.7.1)
Executive Summary - this app could stand a careful documenting from start to finish on a fresh Ubuntu system not used by a knowledgeable JavaScript person.
Wondering what your thoughts on adding a brief description of the resource so people know what that link will give them.
Other things to possibly add to the JSON might be:
Just some ideas on extending your useful project.
I am one of the co-founders of PassiveTotal, now owned by RiskIQ. Our platform is a great resource for passive DNS, WHOIS, SSL certificates, web crawling and more. RiskIQ focuses on surfacing digital risk by web crawlers and Internet scanning. The PassiveTotal platform is a great way to pivot through the data sets to identify reputation, context and correlate OSINT. Registration is free and all accounts have enough queries to be useful on a daily basis.
Please add to a new International folder.
(Telephone Numbers) > (International) > (Numberway)
{
"id": "<ID>",
"name": "Numberway",
"type": "url",
"url": "https://www.numberway.com/"
},
http://collec.to/
Instagram from 2th June 2016 has blocked the application that allows us to give you the best experience. We apologize for the inconvenience. We ask you to leave your email to receive news about the projec
http://www.picturegr.am/
There is no app configured at that hostname.
Perhaps the app owner has renamed it, or you mistyped the URL.
https://instagr.in/explore/
Not reachable
For contributors, it is not obvious how to assign an ID to a folder
or url
node (or any other, if there are any, for that matter).
Also, it is perhaps a good idea to use GUIDs for the ID fields to have non-conflicting UIDs when creating pull requests.
I tried to use spokeo for people search but found no possibility to see any result for free. There are only paid plans.
When I use the d3 files from the d3js.org website it does not work with your files.
How do we make your files work with d3?
Thanks
Please add to a new International folder.
(Telephone Numbers) > (International) > (Numbering Plans)
{
"id": "<ID>",
"name": "Numbering Plans",
"type": "url",
"url": "https://www.numberingplans.com/?page=analysis&sub=phonenr"
},
scanless - online port scan scraper
IP Address -> Host / Port Discovery:
{
"id": "<ID>",
"name": "scanless (T)",
"type": "url",
"url": "https://github.com/vesche/scanless"
},
usernamecheck[.]com sppears on the Google (un)safe browsing database and if you power through and ignore the warning the site is the Apache "it works" page.
I suggest removing their entry.
Would you consider adding Tinder? (Now has browser based version)
(Dating) > (Tinder)
{
"id": "<ID>",
"name": "Tinder (R)",
"type": "url",
"url": "https://tinder.com/"
},
Hello, i would like to make a suggestion with regards to a resource and a tool to be added to OSINT-Framework. The resource i am referring to is HoneyDB. Which in short is an aggregative threat intel pool. HoneyDB collects and organizes data on malicious hosts from HoneyPy honeypots. They offer a threat feed and provide data visualization services as well.
Secondly i am the author of a command line interface which makes it easier for security researchers to retrieve and process the data provided by HoneyDB. The CLI saves logs of the malicious hosts and allows the user to perform an in-script WHOIS lookup and invoke an Nmap scan as well in order to learn more about a particular target if and when desired.
The name of the CLI i wrote is Mimir and you can find it in it's repo on my Github page.
https://github.com/NullArray/Mimir
With kind regards,
NullArray.
As Shodan, MrLooquer discover new devices on Internet. The difference is that MrLooquer is focus on IPv6 protocol and as Shodan we are pioneering on IPv6 scanning.
MrLooquer: IPv6 Intelligence.
https://mrlooquer.com
MageScan - Magento security scanner.
Domain Name -> Vulnerabilities -> Scanners:
{
"id": "<ID>",
"name": "MageScan",
"type": "url",
"url": "https://magescan.com/"
},
I just created the PR #77 and it was a pain in the ass to find an ID thats not used.
Lets discuss ways to make it easier for people to submit new content as a PR instead of just creating an issue.
Why use python as the HTTP server when http-server could be used instead? One less dependency.
OpSec -> Anonymous Browsing -> Anonymous VPNs -> That One Privacy Guy's VPN Comparison Chart URL should be changed to https://thatoneprivacysite.net/
The README.md only casually mentions the use of python. The very brief statement actually assumes the use of python 2, because the used SimpleHTTPServer
module was merged to the python core with python 3 and is now called http.server
.
Apart from this documentation need, the dependency on python is probably superfluous, because a webserver could also be hosted with node itself (but that's for another issue).
Sent you a tweet on twitter as well. Thought you might look at including this site for historical DNS research.
https://securitytrails.com/
Malicious File Analysis/Hosted Automated Analysis.
Been my goto for a while now. A bit slow lately, but super in depth.
It seems that http://www.boardtracker.com/ should be removed from the list as the site provides no service.
Forums -> Forum Search Engine -> BoardTracker
{
"id": "<ID>",
"name": "Public buckets",
"type": "url",
"url": "https://buckets.grayhatwarfare.com/"
},
Can conduct “password” searches.
(Email Address) > (Breach Data) > (DeHashed)
{
"id": "<ID>",
"name": "DeHashed",
"type": "url",
"url": "https://dehashed.com/"
},
I don't know if you knew about this or not, but since your framework is just static files (no php or anything like that) you can actually use github to host it. It is called github pages that is just letting you know a bit about it. That way you don't have to pay for hosting fee's, if you are, and you can still keep the same domain as well. Just a little suggestion, because that is what we are doing with one of our (repos)[https://github.com/49thSecurityDivision/slideshows). Let me know if you have any other questions! I meant to tell you during the OSINT class, but forgot and just remembered, because of messing around with one of our repos.
I know we already spoke about this but I wanted to make another issue to remind you and because I'm not using Github enough.
Check out https://crt.sh/?cn=%25.nsa.gov!! Search for other domains too! apple.com?
Just saw that the http://www.i-intelligence.eu/wp-content/uploads/2016/11/2016_November_Open-Source-Intelligence-Tools-and-Resources-Handbook.pdf document was released with a HUGE number of tools and websites in it.
Just putting this here for someone to go through and scrape content from it.
SameIP.org - Reverse IP Lookup
IP Address -> Neighbor Domains:
{
"id": "<ID>",
"name": "SameIP",
"type": "url",
"url": "https://sameip.org/"
},
{
"id": "<ID>",
"name": "CloudScraper (T)",
"type": "url",
"url": "https://github.com/jordanpotti/cloudscraper"
},
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.