blacktop / docker-zeek Goto Github PK
View Code? Open in Web Editor NEWZeek IDS Dockerfile
License: MIT License
Zeek IDS Dockerfile
License: MIT License
Hi blacktop.
How can we monitor several interfaces at the same time.
"Zeek -i" accepts only one interface
Thanks for your works :)
Is it normal that the container runs at almost 300% cpu (see screenshot from portainer). running it with this command line:
docker run -d --name zeek --cap-add=NET_RAW --net=host -v /opt/threathunt/zeek/logs:/pcap:rw -v /opt/threathunt/zeek/local.zeek:/usr/local/zeek/share/zeek/site/local.zeek blacktop/zeek -i ens33 local "Site::local_nets += { 172.16.100.0/24 }"
Cheers,
Luk
Without the --branch @git clone, with the master, building goes well.
I think because of recent changes in zeek.
With your current version
I get this build error:
-- Could NOT find LibMMDB (missing: LibMMDB_LIBRARY LibMMDB_INCLUDE_DIR)
-- Could NOT find LibKrb5 (missing: LibKrb5_LIBRARY LibKrb5_INCLUDE_DIR)
-- Could NOT find GooglePerftools (missing: GooglePerftools_LIBRARIES GooglePerftools_LIBRARIES_DEBUG GooglePerftools_INCLUDE_DIR)
-- Could NOT find tcmalloc (missing: GooglePerftools_LIBRARIES)
CMake Error at CMakeLists.txt:199 (find_package):
By not providing "FindFTS.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "FTS", but
CMake did not find one.
Could not find a package configuration file provided by "FTS" with any of
the following names:
FTSConfig.cmake
fts-config.cmake
Add the installation prefix of "FTS" to CMAKE_PREFIX_PATH or set "FTS_DIR"
to a directory containing one of the above files. If "FTS" provides a
separate development package or SDK, be sure it has been installed.
-- Configuring incomplete, errors occurred!
See also "/tmp/zeek/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/zeek/build/CMakeFiles/CMakeError.log".
The command '/bin/sh -c echo "===> Compiling zeek..." && cd /tmp/zeek && CC=clang ./configure --prefix=/usr/local/bro --disable-broker-tests --disable-broctl --disable-auxtools && make && make install' returned a non-zero code: 1
Hi guys is it possible to provide builds compatible with raspberryPi?
restart: on-failure:5
Because of the Final brownout the git address in line 56 needs to be corrected to https://github.com/hosom/file-extraction
https://github.com/blacktop/docker-zeek/blob/master/docker-compose.live.yml
networks:
elastinet:
Hello,
Does anyone know how to integrate this docker with the Kafka?
Thanks
filebeat | 2022-01-23T13:11:58.347Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.347Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.347Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.347Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat | 2022-01-23T13:11:58.348Z ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
Hi!
I was trying to load a custom local.zeek
config script by mounting it at /usr/local/share/zeek/site/local.zeek
as described in the documentation. The changes introduced by the script were not working, so I checked the zeek logs and observed that the local.zeek
file being loaded was located at /usr/local
/zeek/
share/zeek/site/local.zeek
. After mounting my local.zeek
file at this location, Zeek loaded it and the changes were applied.
Hi
Thanks for this Project for combining ELK with Zeek
Can you please explain for monitoring specfic ethernet port in docker-compose.elastic.yml file as you have given for pcap file
Thanks you for the same
Thanks
Hello,
Thank you for your work but after following your explanation for the deployment of Zeek by the docker-compose file I have the Zeek container that starts and stops after errors, here below the log of the Zeek container.
Thanks for your help.
$ git clone --depth 1 https://github.com/blacktop/docker-zeek.git
$ cd docker-zeek
$ docker-compose -f docker-compose.elastic.yml up -d kibana
wait a few minutes for "kibana" to start
$ docker-compose -f docker-compose.elastic.yml up -d filebeat
$ docker-compose -f docker-compose.elastic.yml up zeek
wait a little while for filebeat to consume all the logs
$ open http://localhost:5601/app/kibana
zeek_1
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 49: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 49: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 49: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 49: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 49: no such index (Notice::tmp_notice_storage[Notice::uid])
1295981840.989753 expression error in /usr/local/zeek/share/zeek/policy/frameworks/notice/extend-email/hostnames.zeek, line 39: no such index (Notice::tmp_notice_storage[Notice::uid])
-- Found SubnetTree: build from source aux/pysubnettree
CMake Error at aux/broctl/cmake/InstallShellScript.cmake:44 (message):
Absolute path to interpreter 'bash' not found, failed to configure shell
script: /tmp/zeek/aux/broctl/bin/archive-log
Call Stack (most recent call first):
aux/broctl/CMakeLists.txt:88 (InstallShellScript)
Please let me know how to build the docker image with broctl
-
name: Docker Hub Description
uses: peter-evans/dockerhub-description@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
repository: blacktop/docker-zeek
Hey! This was extremely helpful for me.. as i was trying to integrate zeek, kibana and elasticsearch.
I am now trying to push this into a kubernetes cluster. Do you have any steps for this? i am an amateur and i am trying this for my master thesis project.
I also wanted to know how to add custom scripts into this zeek docker..?
Configure alerts and stuff?
Could you help me out??
Thanks in advance!!!
:) Cheers! keep up the good work
Hi,
It still compiles but there are some openssl issues.
After running bro I get this error.
Error loading shared library libssl.so.1.0.0: No such file or directory (needed by /usr/local/bro/bin/bro)
Error loading shared library libcrypto.so.1.0.0: No such file or directory (needed by /usr/local/bro/bin/bro)
Error loading shared library libssl.so.1.0.0: No such file or directory (needed by /usr/local/bro/lib/libcaf_openssl.so.0.16.3)
Error loading shared library libcrypto.so.1.0.0: No such file or directory (needed by /usr/local/bro/lib/libcaf_openssl.so.0.16.3)
Regards,
Daniel
I'm try to use docker-compose-elastic but fails to build :(
Can you help me?
[ 27%] Building CXX object CMakeFiles/Zeek-AF_Packet.linux-x86_64.dir/src/Plugin.cc.o
In file included from /tmp/zeek/auxil/zeek-af_packet-plugin/src/Plugin.cc:3:
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:28:55: error: expected class-name before '{' token
28 | class AF_PacketSource : public zeek::iosource::PktSrc {
| ^
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:46:9: error: 'PktSrc' does not name a type; did you mean 'pktsrc'? 46 | static PktSrc* InstantiateAF_Packet(const std::string& path, bool is_live);
| ^~~~~~
| pktsrc
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:52:33: error: 'zeek::Packet' has not been declared
52 | virtual bool ExtractNextPacket(zeek::Packet* pkt);
| ^~~~
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:56:26: error: 'Stats' has not been declared
56 | virtual void Statistics(Stats* stats);
| ^~~~~
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:59:2: error: 'Properties' does not name a type
59 | Properties props;
| ^~~~~~~~~~
/tmp/zeek/auxil/zeek-af_packet-plugin/src/AF_Packet.h:60:2: error: 'Stats' does not name a type
60 | Stats stats;
| ^~~~~
/tmp/zeek/auxil/zeek-af_packet-plugin/src/Plugin.cc: In member function 'virtual zeek::plugin::Configuration plugin::Zeek_AF_Packet::Plugin::Configure()':
/tmp/zeek/auxil/zeek-af_packet-plugin/src/Plugin.cc:12:19: error: expected type-specifier before '::' token
12 | AddComponent(new ::zeek::iosource::PktSrcComponent("AF_PacketReader", "af_packet", ::zeek::iosource::PktSrcComponent::LIVE, ::zeek::iosource::pktsrc::AF_PacketSource::InstantiateAF_Packet));
| ^~
make[3]: *** [CMakeFiles/Zeek-AF_Packet.linux-x86_64.dir/build.make:102: CMakeFiles/Zeek-AF_Packet.linux-x86_64.dir/src/Plugin.cc.o] Error 1
make[3]: Leaving directory '/tmp/zeek/auxil/zeek-af_packet-plugin/build'
make[2]: Leaving directory '/tmp/zeek/auxil/zeek-af_packet-plugin/build'
make[1]: Leaving directory '/tmp/zeek/auxil/zeek-af_packet-plugin/build'
make[2]: *** [CMakeFiles/Makefile2:134: CMakeFiles/Zeek-AF_Packet.linux-x86_64.dir/all] Error 2
make[1]: *** [Makefile:172: all] Error 2
make: *** [Makefile:13: build-it] Error 2
The command '/bin/sh -c echo "===> Compiling af_packet plugin..." && cd /tmp/zeek/auxil/ && git clone https://github.com/J-Gras/zeek-af_packet-plugin.git && cd /tmp/zeek/auxil/zeek-af_packet-plugin && CC=clang ./configure --with-kernel=/usr --zeek-dist=/tmp/zeek && make -j 2 && make install && /usr/local/zeek/bin/zeek -NN Zeek::AF_Packet' returned a non-zero code: 2
ERROR: Service 'zeek' failed to build
This is a very useful zeek container project, I want to apply it to my project. I have a self-built docker network (spark-net), which contains a Kafka container and a Spark container.I want to use zeek to ingest real-time traffic data for some analysis, but I don't know how to make zeek communicate with Kafka in spark-net while being able to listen to external traffic.
Use docker network list
to view, the DRIVER
of the self-built network is bridge
, and the SCOPE
is local
Use ifconfig -a
to view the host network card as follows:
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
inet6 fe80::42:22ff:fe87:6f8a prefixlen 64 scopeid 0x20<link>
ether 02:42:22:87:6f:8a txqueuelen 0 (Ethernet)
RX packets 827091 bytes 4974746136 (4.9 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 963491 bytes 1691371345 (1.6 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.66.7 netmask 255.255.255.0 broadcast 192.168.66.255
inet6 2408:8210:783e:e473:9b4:dee2:734c:5fb4 prefixlen 64 scopeid 0x0<global>
The configuration of local.zeek uses it
I changed ["metadata.broker.list"] = "kafka:9092"
to ["metadata.broker.list"] = "localhost:9092"
Dockerfile uses it
I have tried the following:
1. Use docker-compose to integrate the zeek container into spark-net to start
I refer to docker-compose.live.yml, part of the compose is:
kafka:
container_name: kafka
image: kafka:latest
build: kafka/.
volumes:
- ./kafka/server.properties_template:/opt/kafka/config/server.properties_template
- ./kafka/entrypoint.sh:/entrypoint/entrypoint.sh
depends_on:
- "zookeeper"
restart: always
ports:
- "9092:9092"
networks:
- spark-net
zookeeper:
container_name: zookeeper
image: zookeeper:latest
build: zookeeper/.
depends_on:
- "logstash"
restart: always
ports:
- "2181:2181"
networks:
- spark-net
zeek:
depends_on:
- kafka
build: ./zeek
image: blacktop/zeek:elastic
volumes:
- ./pcap:/pcap
cap_add:
- NET_RAW
network_mode: "host"
command: -i af_packet::eno1 local
networks:
spark-net:
But after starting the network, the status of the zeek container is Restarting (0) About a minute ago
, apparently it is not working properly
2. Start the zeek container by command without using docker-compose
This method is divided into two cases:
①Start the zeek container in host mode (I refer to Capture Live Traffic)
docker run --rm \
--cap-add=NET_RAW \
--net host \
-v `pwd`:/pcap:rw blacktop/zeek:kafka \
-i af_packet::eno1 local\
I get the following error:
listening on eno1
WARNING: No Site::local_nets have been defined. It's usually a good idea to define your local networks.
%3|1651201492.014|FAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT)
%3|1651201492.014|ERROR|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: 1/1 brokers are down
%3|1651201492.015|ERROR|rdkafka#producer-1| [thrd:app]: rdkafka#producer-1: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT)
%3|1651201492.019|FAIL|rdkafka#producer-2| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT)
%3|1651201492.019|ERROR|rdkafka#producer-2| [thrd:kafka:9092/bootstrap]: 1/1 brokers are down
%3|1651201492.030|FAIL|rdkafka#producer-3| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5006ms in state CONNECT)
%3|1651201492.030|ERROR|rdkafka#producer-3| [thrd:kafka:9092/bootstrap]: 1/1 brokers are down
%3|1651201493.015|ERROR|rdkafka#producer-2| [thrd:app]: rdkafka#producer-2: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT)
%3|1651201493.015|ERROR|rdkafka#producer-3| [thrd:app]: rdkafka#producer-3: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5006ms in state CONNECT)
%3|1651201499.015|FAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5004ms in state CONNECT, 1 identical error(s) suppressed)
%3|1651201499.016|ERROR|rdkafka#producer-1| [thrd:app]: rdkafka#producer-1: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5004ms in state CONNECT, 1 identical error(s) suppressed)
%3|1651201499.019|FAIL|rdkafka#producer-2| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT, 1 identical error(s) suppressed)
%3|1651201499.029|FAIL|rdkafka#producer-3| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5005ms in state CONNECT, 1 identical error(s) suppressed)
%3|1651201500.016|ERROR|rdkafka#producer-2| [thrd:app]: rdkafka#producer-2: kafka:9092/bootstrap: Failed to resolve 'kafka:9092': Try again (after 5003ms in state CONNECT, 1 identical error(s) suppressed)
It seems that zeek can listen to eno1 but cannot access Kafka inside spark-net
②Start after connecting the zeek container with spark-net via --link
docker run --rm \
--cap-add=NET_RAW \
--net elk_spark_elastinet \
--link elksj-kafka:localhost\
-v `pwd`:/pcap:rw blacktop/zeek:kafka \
-i af_packet::eno1 local\
After running I get the error message:fatal error: problem with interface af_packet::eno1 (No such device)
It seems that within spark-net, zeek cannot listen to the external network card
So I tried listening to docker0 (and docker0's ip), but the result is the same
What should I do to achieve real-time monitoring of the host (or any network card) in the container network? do you have any good advice?
Anyone have any idea can let me know, please😔
Hi
I tried to run zeek/kafka integration on on macos Mojave.
When I run docker run --rm -v pwd
:/pcap -v pwd
/local.zeek:/usr/local/zeek/share/zeek/site/local.zeek --link kafka:localhost blacktop/zeek:kafka -P -r heartbleed.pcap local "Site::local_nets += { 192.168.11.0/24 }",
I got the following error:
warning in /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.zeek, line 39: deprecated (bro_init): Remove in v3.1: use zeek_init
fatal error in , line 1: can't update DNS cache
I cannot figure out why. Can sombebody help?
thanks
Hi,
When following the directions on https://github.com/blacktop/docker-zeek/blob/master/docs/elastic.md under "Use LIVE Traffic", the blacktop/filebeat part fails:
klaus@docker:~$ docker run --init --rm -it -v `pwd`:/pcap --link kibana --link elasticsearch blacktop/filebeat -e
Unable to find image 'blacktop/filebeat:latest' locally
latest: Pulling from blacktop/filebeat
9d48c3bd43c5: Already exists
fc3cebc17991: Pull complete
5cc76213ba9d: Pull complete
3e7376168fbf: Pull complete
b861f6c07f10: Pull complete
3946a80b0229: Pull complete
Digest: sha256:437128d2d77fe4eea77c10852a6eac569ec9931662a7bb3ce49a131fe39a6683
Status: Downloaded newer image for blacktop/filebeat:latest
===> Waiting on elasticsearch(elasticsearch:9200) to start...
Elasticsearch is ready!
===> Waiting for Kibana(kibana:5601) to start...Kibana is ready!
===> Setting up filebeat...
2019-10-16T08:38:28.342Z INFO instance/beat.go:606 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-10-16T08:38:28.351Z INFO instance/beat.go:614 Beat ID: d492666b-2800-425b-a92c-16a3f383e5f7
2019-10-16T08:38:28.352Z INFO [beat] instance/beat.go:902 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "d492666b-2800-425b-a92c-16a3f383e5f7"}}}
2019-10-16T08:38:28.352Z INFO [beat] instance/beat.go:911 Build info {"system_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:30:50.000Z", "version": "7.3.1"}}}
2019-10-16T08:38:28.352Z INFO [beat] instance/beat.go:914 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.4"}}}
2019-10-16T08:38:28.354Z INFO [beat] instance/beat.go:918 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-10-13T17:33:29Z","containerized":true,"name":"c86de9f6e08a","ip":["127.0.0.1/8","172.17.0.9/16"],"kernel_version":"4.19.0-6-amd64","mac":["02:42:ac:11:00:09"],"os":{"family":"","platform":"alpine","name":"Alpine Linux","version":"","major":0,"minor":0,"patch":0},"timezone":"UTC","timezone_offset_sec":0}}}
2019-10-16T08:38:28.356Z INFO [beat] instance/beat.go:947 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 12, "ppid": 6, "seccomp": {"mode":"filter","no_new_privs":false}, "start_time": "2019-10-16T08:38:27.760Z"}}}
2019-10-16T08:38:28.357Z INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.3.1
2019-10-16T08:38:28.357Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.3.1' as ILM is enabled.
2019-10-16T08:38:28.358Z INFO elasticsearch/client.go:170 Elasticsearch url: http://elasticsearch:9200
2019-10-16T08:38:28.359Z INFO [publisher] pipeline/module.go:97 Beat name: c86de9f6e08a
2019-10-16T08:38:28.366Z INFO beater/filebeat.go:92 Enabled modules/filesets: zeek (dns, files, http, notice, ssl, connection)
2019-10-16T08:38:28.380Z INFO elasticsearch/client.go:170 Elasticsearch url: http://elasticsearch:9200
2019-10-16T08:38:28.632Z INFO elasticsearch/client.go:743 Attempting to connect to Elasticsearch version 7.0.1
2019-10-16T08:38:28.639Z ERROR elasticsearch/elasticsearch.go:260 Error connecting to Elasticsearch at http://elasticsearch:9200: Connection marked as failed because the onConnect callback failed: This Beat requires the default distribution of Elasticsearch. Please install the default distribution of Elasticsearch from elastic.co, or install the oss-only distribution of beats
2019-10-16T08:38:28.639Z ERROR instance/beat.go:877 Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://elasticsearch:9200: Connection marked as failed because the onConnect callback failed: This Beat requires the default distribution of Elasticsearch. Please install the default distribution of Elasticsearch from elastic.co, or install the oss-only distribution of beats]
Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch http://elasticsearch:9200: Connection marked as failed because the onConnect callback failed: This Beat requires the default distribution of Elasticsearch. Please install the default distribution of Elasticsearch from elastic.co, or install the oss-only distribution of beats]
So apparently there's a mismatch here as the filebeat in the docker is the non OSS version. Or am I wrong? Are there other way to fix it than you putting another version of filebeat in the dockerbuild?
Thanks
/klaus
blacktop/zeek:elastic
The build with spicy will be huge size. It will be a problem.
I tried to Run Bro with the Kafka plugin,there is sth wrong tips:
warning in /usr/local/zeek/lib/zeek/plugins/APACHE_KAFKA/scripts/Apache/Kafka/./logs-to-kafka.zeek, line 39: deprecated (bro_init): Remove in v3.1: use zeek_init
fatal error in , line 1: can't update DNS cache
Is the actual zeekctl
Dockerfile installing/building zeekctl ? I don't see much difference comparing to the zeek Dockerfile and I can't find it inside /usr/local/zeek/bin/
If I execute bash shell in the container.
Hi,
I encountered the following error when running the docker-zeek for elastic using docker-compose.elastic.yml
WARNING: Image for service zeek was built because it did not already exist. To rebuild this image you must use docker-compose build
or docker-compose up --build
.
docker-zeek_elasticsearch_1 is up-to-date
docker-zeek_kibana_1 is up-to-date
docker-zeek_filebeat_1 is up-to-date
Creating docker-zeek_zeek_1 ... done
Attaching to docker-zeek_zeek_1
zeek_1 | Error relocating /usr/local/zeek/lib/libbroker.so.2: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/lib/libbroker.so.2: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/lib/libcaf_core.so.0.17.4: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/lib/libcaf_io.so.0.17.4: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/lib/libcaf_openssl.so.0.17.4: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/bin/zeek: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
zeek_1 | Error relocating /usr/local/zeek/bin/zeek: _ZNSt7__cxx1118basic_stringstreamIcSt11char_traitsIcESaIcEEC1Ev: symbol not found
docker-zeek_zeek_1 exited with code 127
Please advise. Thank you.
@blacktop If it is okay I can create PR on that ?
Would it be possible to include Corelight's community-id package this would help correlate logs with suricata.
https://dactiv.llc/blog/enable-zeek-community-id/
https://github.com/corelight/zeek-community-id
great docker container for zeek, I'm having trouble getting the json policy output to kick in by adding my own local.zeek. Does this mean the container needs to be built and compiled with the dockerfile in order to get the policy changes to take effect?
sudo docker run -d --cap-add=NET_RAW --net=host -v <pwd>/local.zeek:/usr/local/zeek/share/zeek/site/local.zeek -v <pwd>:/pcap:rw blacktop/zeek -i ens160
Hi There,
I was wondering if there might we a way to integrate your zeek image in the project suricata-filestash-elk-docker here on github.
I'd like both tools but not have to double up on elk containers.
Do you have customizations on your images that would prevent this?
Hi man,
you rock! there is a way of making a docker compose file that will create Docker container with network traffic from Zeek and Snort, and the data from these 2 tools to be taken into Kafka and from Kafka into ELK?
Thank you again!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.