Comments (6)
I'd be happy to take a pass at this. I'm excited to see zgrab2 and I like the modular approach. This seems a lot easier to write modules for than the previous version of zgrab. 😄
I'd be happy to take on MongoDB and other nosql databases/document stores (e.g. elasticsearch) as well as key-value stores (memcached/redis). I'll make issues for them before working on them for tracking.
I do have a quick question though: What kind of information is important to Censys? MongoDB without authentication can offer information about the host, such as database size compared to available space. It also contains info about the various collections and more. Shodan collects this information but I wasn't sure if it'd be important to Censys to collect it. It would result in more HTTP round trips, which might add overhead for mass ipv4 scans.
Once I know what information is important to y'all, I'll get to work. My plan is to work through an initial module, taking notes on how it's implemented. I'll put those notes in a blog post to encourage other participation, and we can consider if the notes might be useful in the wiki here.
from zgrab2.
@jordan-wright I had not started on MongoDB yet, so no worries about duplicated labor.
from zgrab2.
I'd be happy to take a pass at this. I'm excited to see zgrab2 and I like the modular approach. This seems a lot easier to write modules for than the previous version of zgrab. 😄
Awesome!
I do have a quick question though: What kind of information is important to Censys? MongoDB without authentication can offer information about the host, such as database size compared to available space. It also contains info about the various collections and more. Shodan collects this information but I wasn't sure if it'd be important to Censys to collect it. It would result in more HTTP round trips, which might add overhead for mass ipv4 scans.
We generally adopt the approach of "collect as much information as possible without needing to open a second TCP socket or logging in". I expect performance won't be that large of an issue here, since there shouldn't be nearly as many Mongo hosts as there are HTTPS hosts. We can always tone down some collection if needed. We'll have a better sense of this once we start seeing some initial output, etc as the module is written.
Tagging @justinbastress, who I think may have also been looking into this scanner.
from zgrab2.
"collect as much information as possible without needing to open a second TCP socket or logging in"
This is a perfectly reasonable approach. And I suppose nothing says we can't add more information later. Really, just knowing the number of mongodb hosts is a huge lift. Collection, system, etc. information can be made available later if the cycles permit it 🤷♂️ Getting enriched information at the app layer is great, but alluding to the "one TCP socket" approach it definitely comes with a performance cost.
And @justinbastress please let me know if I'm stepping over work that's already done. The last thing I want to do is repeat work, so I can always start somewhere else, like Redis or Elasticsearch.
from zgrab2.
Out of curiosity, what's the current stance on leveraging external libraries to gather some service specific data? For example, MongoDB has a weird wire protocol that might be a bit tricky to implement adhoc if we wanted to do it ourselves.
The option exists to use mgo
to handle setting up the connection and pulling data, but it adds a dependency.
The only reason I ask is because it looks like the option for MySQL scanning was implemented in a custom library in zgrab.
from zgrab2.
MongoDB scanner implemented.
from zgrab2.
Related Issues (20)
- Handshake failure in http module HOT 2
- hosts http-req HOT 1
- Zgrab2 output not come
- test errors lib/output/test/process_test.go
- test failures lib/ssh
- test failures modules/redis
- modules/fox - go vet warning causing test failures
- HTTP body content is truncated in some cases despite setting max-size to a high value HOT 5
- Support opt-out (or make zgrab opt-in) HOT 2
- Question Regarding Modification of ZGrab 2.0 Output Content HOT 3
- Outdated Installation Guide HOT 1
- Is this repo unmaintained? HOT 2
- url error HOT 1
- Support user-specified DNS server for dialer
- Zgrab2 cannot automatically load the root certificate of the operating system HOT 1
- Remove EoL dependency on Python2
- Please bump the Golang version to a modern one HOT 6
- ZGrab is adding a trailing slash `/` to URL's with http scans HOT 2
- ZGrab performance issue with http? HOT 8
- How to setup no retries for tls module? (i/o timeout) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zgrab2.