ahlashkari / dohlyzer Goto Github PK
View Code? Open in Web Editor NEWDoHlyzer is a DNS over HTTPS (DoH) traffic flow generator and analyzer for anomaly detection and characterization.
DoHlyzer is a DNS over HTTPS (DoH) traffic flow generator and analyzer for anomaly detection and characterization.
C:\Users\hp\Desktop\DoHlyzer-master\meter>python3 dohlyzer.py -n eth0 -c ./output.csv
Traceback (most recent call last):
File "dohlyzer.py", line 8, in
from meter.flow_session import *
ModuleNotFoundError: No module named 'meter'
reading from file ./MaliciousDoH-dns2tcp-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T21_54_32.055088.pcap, link-type LINUX_SLL (Linux cooked v1)
Packet count: 969
Garbage Collection Began. Flows = 2
/usr/lib/python3/dist-packages/scipy/stats/stats.py:263: RuntimeWarning: The input array could not be properly checked for nan values. nan values will b
e ignored.
warnings.warn("The input array could not be properly checked for nan "
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow.py", line 80, in get_data
'PacketTimeMode': packet_time.get_mode(),
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/features/packet_time.py", line 114, in get_mode
mode = float(stat.mode(self._get_packet_times())[0])
File "/usr/lib/python3/dist-packages/scipy/stats/stats.py", line 458, in mode
if a.dtype == object and np.nan in set(a.ravel()):
TypeError: unhashable type: 'EDecimal'
mohittembe@DESKTOP-8FG1VFR:/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter$ python3 dohlyzer.py -f ./MaliciousDoH-dns2tcp
-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap -c ./output.csv
reading from file ./MaliciousDoH-dns2tcp-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap, link-type LINUX_SLL (Linux cooked v1)
Packet count: 945
Garbage Collection Began. Flows = 2
/usr/lib/python3/dist-packages/scipy/stats/stats.py:263: RuntimeWarning: The input array could not be properly checked for nan values. nan values will b
e ignored.
warnings.warn("The input array could not be properly checked for nan "
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow.py", line 80, in get_data
'PacketTimeMode': packet_time.get_mode(),
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/features/packet_time.py", line 114, in get_mode
mode = float(stat.mode(self._get_packet_times())[0])
File "/usr/lib/python3/dist-packages/scipy/stats/stats.py", line 458, in mode
if a.dtype == object and np.nan in set(a.ravel()):
TypeError: unhashable type: 'EDecimal'
mohittembe@DESKTOP-8FG1VFR:/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter$ python3 dohlyzer.py -f ./MaliciousDoH-dns2tcp
-Pcap-001_600 -c ./output.csv
tcpdump: error reading dump file: Is a directory
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/scapy/sendrecv.py", line 882, in _run
sniff_sockets[PcapReader(
File "/usr/local/lib/python3.8/dist-packages/scapy/utils.py", line 975, in call
raise Scapy_Exception(
scapy.error.Scapy_Exception: No data could be read!
mohittembe@DESKTOP-8FG1VFR:/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter$ python3 dohlyzer.py -f ./MaliciousDoH-dns2tcp
-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap -c ./output.csv
reading from file ./MaliciousDoH-dns2tcp-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap, link-type LINUX_SLL (Linux cooked v1)
Packet count: 945
Garbage Collection Began. Flows = 2
/usr/lib/python3/dist-packages/scipy/stats/stats.py:263: RuntimeWarning: The input array could not be properly checked for nan values. nan values will b
e ignored.
warnings.warn("The input array could not be properly checked for nan "
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow.py", line 80, in get_data
'PacketTimeMode': packet_time.get_mode(),
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/features/packet_time.py", line 114, in get_mode
mode = float(stat.mode(self._get_packet_times())[0])
File "/usr/lib/python3/dist-packages/scipy/stats/stats.py", line 458, in mode
if a.dtype == object and np.nan in set(a.ravel()):
TypeError: unhashable type: 'EDecimal'
mohittembe@DESKTOP-8FG1VFR:/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter$ python3 dohlyzer.py -f ./MaliciousDoH-dns2tcp
-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap -c ./output.csv
reading from file ./MaliciousDoH-dns2tcp-Pcap-001_600/dns2tcp_tunnel_1111_doh1_2020-03-31T22_17_01.945354.pcap, link-type LINUX_SLL (Linux cooked v1)
Packet count: 945
Garbage Collection Began. Flows = 2
/usr/lib/python3/dist-packages/scipy/stats/stats.py:263: RuntimeWarning: The input array could not be properly checked for nan values. nan values will b
e ignored.
warnings.warn("The input array could not be properly checked for nan "
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/flow.py", line 80, in get_data
'PacketTimeMode': packet_time.get_mode(),
File "/mnt/c/Users/mohit/Onedrive/Desktop/studentjob/project/DOHmeter/DoHlyzer/meter/features/packet_time.py", line 114, in get_mode
mode = float(stat.mode(self._get_packet_times())[0])
File "/usr/lib/python3/dist-packages/scipy/stats/stats.py", line 458, in mode
if a.dtype == object and np.nan in set(a.ravel()):
TypeError: unhashable type: 'EDecimal'
I am getting this error while uploading the pcap file. Can you please help me with this?
This is the link for pcap file I am trying to upload
http://205.174.165.80/CICDataset/DoHBrw-2020/Dataset/MaliciousDoH-dns2tcp-Pcaps/
The import in this file should be
from meter.features.context.packet_direction import PacketDirection not from meter.features.context import packet_direction
and the line 33 should be if direction == PacketDirection.FORWARD:
When I ryn the meter/dohlyzer.py file I get the error
"" File "./dohlyzer.py", line 8, in
from meter.flow_session import generate_session_class
ModuleNotFoundError: No module named 'meter'
""
I tried to add the meter folder to the packages but I still get errors
Any idea please
I've tried Python 2.7, 3.5, 3.6, 3.7, and 3.8 each with its own issues.
Python 2.7, Python 3.5 & Python 3.8 are not relevant - due to syntax and unsatisfied requirements.
All python versions producing the following exception:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/ubuntu/Desktop/DoHlyzer-master/venv36/lib/python3.6/site-packages/scapy/sendrecv.py", line 836, in _run
*session_args, **session_kwargs)
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow_session.py", line 31, in __init__
super(FlowSession, self).__init__(None, True, *args, **kwargs)
TypeError: __init__() got multiple values for argument 'prn'
As no use is done with args and kwargs at FlowSession, we can remove the asterisks from the super constructor:
super(FlowSession, self).__init__(None, True, *args, **kwargs)
Not a good solution, but it's working.
After "fixing" the above issue, with Python 3.6 we get the following exception:
Packet count: 10000
Garbage Collection Began. Flows = 191
Garbage Collection Finished. Flows = 191
Packet count: 20000
Garbage Collection Began. Flows = 485
/home/ubuntu/Desktop/DoHlyzer-master/venv36/lib/python3.6/site-packages/scipy/stats/stats.py:269: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
"values. nan values will be ignored.", RuntimeWarning)
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/ubuntu/Desktop/DoHlyzer-master/venv36/lib/python3.6/site-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow.py", line 80, in get_data
'PacketTimeMode': packet_time.get_mode(),
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/features/packet_time.py", line 116, in get_mode
mode = float(stat.mode(self._get_packet_times())[0])
File "/home/ubuntu/Desktop/DoHlyzer-master/venv36/lib/python3.6/site-packages/scipy/stats/stats.py", line 469, in mode
if a.dtype == object and np.nan in set(a.ravel()):
TypeError: unhashable type: 'EDecimal'
With Python 3.7 we have the following exception:
Packet count: 10000
Garbage Collection Began. Flows = 191
Garbage Collection Finished. Flows = 191
Packet count: 20000
Garbage Collection Began. Flows = 485
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/home/ubuntu/Desktop/DoHlyzer-master/venv37/lib/python3.7/site-packages/scapy/sendrecv.py", line 989, in _run
session.on_packet_received(p)
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow_session.py", line 105, in on_packet_received
self.garbage_collect(packet.time)
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow_session.py", line 119, in garbage_collect
data = flow.get_data()
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/flow.py", line 78, in get_data
'PacketTimeMean': packet_time.get_mean(),
File "/home/ubuntu/Desktop/DoHlyzer-master/meter/features/packet_time.py", line 94, in get_mean
mean = numpy.mean(self._get_packet_times())
File "<__array_function__ internals>", line 6, in mean
File "/home/ubuntu/Desktop/DoHlyzer-master/venv37/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 3420, in mean
out=out, **kwargs)
File "/home/ubuntu/Desktop/DoHlyzer-master/venv37/lib/python3.7/site-packages/numpy/core/_methods.py", line 190, in _mean
ret = ret / rcount
File "/home/ubuntu/Desktop/DoHlyzer-master/venv37/lib/python3.7/site-packages/scapy/utils.py", line 84, in __truediv__
return EDecimal(Decimal.__truediv__(self, Decimal(other), **kwargs))
TypeError: conversion from numpy.int64 to Decimal is not supported
Python 3.8 couldn't have worked as well:
ERROR: Could not find a version that satisfies the requirement tensorflow==2.1.0
ERROR: No matching distribution found for tensorflow==2.1.0
Tested on Windows 10 & Ubuntu 20.04
Command-line: (venv3X) ubuntu@ubuntu-vm:~/Desktop/DoHlyzer-master$ python meter/dohlyzer.py -f '/home/ubuntu/Desktop/dump_00005_20200113142226.pcap' -c ./output.csv
It would help if you could write what Python interpreter version you have used.
Tnx :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.