In this project, we build a real-time system which is able to capture network intrusion and then predict its type of attack by an AI model.
- Viet-Sang Nguyen
- Phuong-Hoa Nguyen
- Ngoc-Nhat-Huyen Tran
sudo ./install.sh
Kafka server needs this one. It will run on localhost, port 2181
kafka_2.13-2.4.1/bin/zookeeper-server-start.sh kafka_2.13-2.4.1/config/zookeeper.properties
It will run on localhost, port 9092. When we publish messages to this server through producers, messages will be stored on /tmp/kafka-logs/
kafka_2.13-2.4.1/bin/kafka-server-start.sh kafka_2.13-2.4.1/config/server.properties
brew install snort
Snort needs a config file (snort.config) and a folder to store log. Normally, they are stored in /etc/snort/snort.config and /var/logs/snort.
In this project, we store them in folder snort and use the full paths to point where they are.
In this file, we point to the file rules/icmp.rules
include rules/icmp.rules
We write some rules to capture packets in config file. In this example, sort will alert all ping packets.
alert icmp any any -> any any (msg:"ICMP Packet"; sid:477; rev:3;)
snort -A console -q -c ~/.../src/snort/snort.config -b -i en0 -L ~/.../src/snort/logs/log.pcap
ping google.com
python3 producer_pcap.py
This consumer retrieves pcaps from kafka, then transform to readable data thanks to Zeek. After that, KDD99 data is generated by spark. TensorFlow models then predict whether data is normal or attacked. The results are sent back to kafka.
python3 consumer_spark.py
This consumer retrieves and shows predicted data from kafka
python3 consumer_warning.py
Notice: all commands should be run from NIDS-DL-Project folder
python3 dl_src/preprocessing.py
python3 dl_src/train.py
python3 dl_src/test.py