flyballlabs / threatdetectionservice Goto Github PK
View Code? Open in Web Editor NEWThreat Management Platform with Apache Metron as the core engine
License: Apache License 2.0
Threat Management Platform with Apache Metron as the core engine
License: Apache License 2.0
Users are authenticated by the api but agents should have verification as well.
At a minimum, when an agent calls to the api, it should be checked against the database, if their is a match then we know an authenticated user created that agent.
This will alleviate any script injection or zombie agent usage.
This script is responsible for configuring a Metron installation to work with the Threat Management Platform (TMP). The first release is focused on making it work with the Metron Quickstart VM.
This includes but not limited to adding, updating and deleting Agent configuration data. Once the data is added we should be able to call the agent API and return that data.
The Notifications API will be used to notify a SOC User that a Threat was received with a particular Threat Level. The following components will need to be changed:
Metron Threat Intel Configs:
Please use the issue # to commit any changes. This will allow us to track the commits related to this Issue
A user should be able to specify how they want notification data to be sent. Per the following ways"
Email:
-provide an email field
-provide a score level. For example, only send messages when the score is 5 or less
SMS:
The configuration scripts should be Python based with a name of config.py. There should be a script for api and gui components such as api/config.py and gui/config.py. The scripts should provide a place to put central configuration info.
I've created a standard approach based on the flask documentation that all of our entities within our API should implement. Please apply this approach to the User and Agent entities.
The following is in the "Adding an API" section of this document:
https://github.com/flyballlabs/threatdetectionservice/blob/master/CONTRIBUTING.md
This will allow a user to copy and paste and email into the system and then perform facial recognition on a pre-defined set of mugshots
Can not get metron to enrich threat intel via threat intel bulk loader.
When I try the threat intel bulk load cmd zookeeper hangs at connection, see output below:
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=2.6.32-642.6.2.el6.x86_64
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=root
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/root
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/assets-hosts
2017-01-16 02:38:57,726 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=node1:2181 sessionTimeout=90000 watcher=hconnection-0x4671115f0x0, quorum=node1:2181, baseZNode=/hbase-unsecure
2017-01-16 02:38:57,747 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Opening socket connection to server node1/192.168.66.121:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-16 02:38:57,754 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Socket connection established to node1/192.168.66.121:2181, initiating session
2017-01-16 02:38:57,756 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2017-01-16 02:38:57,861 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=node1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase-unsecure/hbaseid
2017-01-16 02:38:59,233 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Opening socket connection to server node1/192.168.66.121:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-16 02:38:59,234 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Socket connection established to node1/192.168.66.121:2181, initiating session
2017-01-16 02:38:59,234 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2017-01-16 02:38:59,334 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=node1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase-unsecure/hbaseid
2017-01-16 02:39:01,153 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Opening socket connection to server node1/192.168.66.121:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-16 02:39:01,153 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Socket connection established to node1/192.168.66.121:2181, initiating session
2017-01-16 02:39:01,155 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
After replacing the ojdbc.jar file mentioned in the error zookeeper client is unable to connect. See below:
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=2.6.32-642.6.2.el6.x86_64
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=root
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/root
2017-01-16 02:38:57,725 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/assets-hosts
2017-01-16 02:38:57,726 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=node1:2181 sessionTimeout=90000 watcher=hconnection-0x4671115f0x0, quorum=node1:2181, baseZNode=/hbase-unsecure
2017-01-16 02:38:57,747 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Opening socket connection to server node1/192.168.66.121:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-16 02:38:57,754 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Socket connection established to node1/192.168.66.121:2181, initiating session
2017-01-16 02:38:57,756 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2017-01-16 02:38:57,861 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=node1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase-unsecure/hbaseid
Add the ability to search and update agents. We are going to add the MAC Address
We need to have an approach that will automatically update the database schema when changes occur at the DB level
Agents need to be searchable through gui.
Need to be able to add agents using gui.
Should be searchable by company and per single agent as well.
Should be able to add agent to database.
In preparation for the upcoming release we need to style the homepage and login page
Needs to have cmds expanded to accept 'provision' as a cmd.
API needs to have this functionality added as well.
Catch Login errors that happen because the API has a failure other then the API Server not being available.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.