vichargrave / espcap Goto Github PK
View Code? Open in Web Editor NEWPacket capture and indexing with Wireshark and Elasticsearch
Home Page: https://vichargrave.github.io/programming/packet-capture-with-wireshark-and-elasticsearch/
Packet capture and indexing with Wireshark and Elasticsearch
Home Page: https://vichargrave.github.io/programming/packet-capture-with-wireshark-and-elasticsearch/
Hi, this is less of an issue and more suggestion - I noticed if you fat finger the node address you have to wait until after data chunking etc before it errors out. I've added the following to the local repo I pulled down:
es = None
if node is not None:
es = Elasticsearch(node)
print("Testing elasticsearch connection")
if es.ping():
print("Ping success")
else:
print("Unable to ping elasticsearch instance on {}\nPlease check and try again".format(node))
sys.exit(1)
A nice clean / easy way to check for a connection issue early on.
I was going to open a PR, but I don't have permission, so raising it here.
If I run:
python3 espcap.py --node=elasticsearch.vocinity.net:9200 --nic=eth1 --count=100 --bpf="udp port 5060"
Things work fine, but if I use --count=0 so I can leave it up, it never saves anything to elastic search.
Hey, I wonder if you have any idea about the problem I am facing:
root@debian:~/espcap# python3.7 src/espcap.py --file=test_pcaps/test_http.pcap --node=localhost:9200
Loading packet capture file(s)
test_pcaps/test_http.pcap
Running as user "root" and group "root". This could be dangerous.
[ERROR] 'timestamp'
[ERROR] [priority,] message string
Traceback (most recent call last):
File "src/espcap.py", line 77, in init_file_capture
helpers.bulk(client=es, actions=index_packets(capture=capture), chunk_size=chunk, raise_on_error=True)
File "/usr/local/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 300, in bulk
for ok, item in streaming_bulk(client, actions, *args, **kwargs):
File "/usr/local/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 212, in streaming_bulk
actions, chunk_size, max_chunk_bytes, client.transport.serializer
File "/usr/local/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 63, in _chunk_actions
for action, data in actions:
File "/root/espcap/src/indexer.py", line 30, in index_packets
timestamp = int(packet['timestamp'])/1000
KeyError: 'timestamp'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "src/espcap.py", line 124, in main
init_file_capture(es=es, tshark=tshark, pcap_files=pcap_files, chunk=chunk)
File "src/espcap.py", line 81, in init_file_capture
syslog.syslog(syslog.LOG_ERR, e)
TypeError: [priority,] message string
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "src/espcap.py", line 141, in <module>
main()
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "src/espcap.py", line 136, in main
syslog.syslog(syslog.LOG_ERR, e)
TypeError: [priority,] message string
tshark: An error occurred while printing packets: Broken pipe.
The command I used:
python3.7 src/espcap.py --file=test_pcaps/test_http.pcap --node=localhost:9200
I issued the command from ~/espcap and my elk-stack (7.x) is running on localhost:9200.
I followed your installation-guide and did everything as you described.
Running on debian 9
Is there some docker image to run this espcap+elasticsearch+kibaba?
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.