Comments (13)
Thank you for pointing this out to me. Yes, I switch all my elastic images over to the OSS versions a while back and forgot to update filebeat. 🤦♂️
Thank you!! All images are building in DockerHUB now, so in a few hours it should work. 👍
Please let me know if there is still an issue.
from docker-zeek.
Hi again
Seems to have had no effect:
docker run --init --rm -it -v `pwd `:/pcap --link kibana --link elasticsearch blacktop/filebeat:7.4.0 -e
Gives me the exact same error. :-(
Thanks for all your work!
/klaus
from docker-zeek.
Apologies, the filebeat image should now be good for 7.4.0
from docker-zeek.
Hi again
Unfortunately, it still doesn't work. Integration or configuration towards zeek seem to be missing..
Can you tell me what's wrong? As I understand it, something is missing from the Filebeat Docker image. Correct?
klaus@docker:~$ docker run --init --rm -it -v `pwd`:/pcap --link kibana --link elasticsearch blacktop/filebeat -e
Unable to find image 'blacktop/filebeat:latest' locally
latest: Pulling from blacktop/filebeat
9d48c3bd43c5: Already exists ef47bfec7715: Pull complete c25e4b428d67: Pull complete c3004d8ecbc2: Pull complete db9be8c78521: Pull complete 4c745d05ed61: Pull complete Digest: sha256:b48d67331c9cc60a6bec7c901101f90f220dd2ce36b244a545796804c478e760 Status: Downloaded newer image for blacktop/filebeat:latest
===> Waiting on elasticsearch(elasticsearch:9200) to start...
Elasticsearch is ready!
===> Waiting for Kibana(kibana:5601) to start...Kibana is ready!
===> Setting up filebeat...
2019-10-20T07:43:13.746Z INFO instance/beat.go:606 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-10-20T07:43:13.755Z INFO instance/beat.go:614 Beat ID: 1746db25-63d0-498f-b50a-2c97ddcaa30a
2019-10-20T07:43:13.756Z INFO [beat] instance/beat.go:902 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "1746db25-63d0-498f-b50a-2c97ddcaa30a"}}}
2019-10-20T07:43:13.756Z INFO [beat] instance/beat.go:911 Build info {"system_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:11:55.000Z", "version": "7.3.1"}}}
2019-10-20T07:43:13.757Z INFO [beat] instance/beat.go:914 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.4"}}} 2019-10-20T07:43:13.758Z INFO [beat] instance/beat.go:918 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-10-17T13:25:15Z","containerized":true,"name":"cfb6d8d1e59e","ip":["127.0.0.1/8","172.17.0.9/16"],"kernel_version":"4.19.0-6-amd64","mac":["02:42:ac:11:00:09"],"os":{"family":"","platform":"alpine","name":"Alpine Linux","version":"","major":0,"minor":0,"patch":0},"timezone":"UTC","timezone_offset_sec":0}}}
2019-10-20T07:43:13.761Z INFO [beat] instance/beat.go:947 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 12, "ppid": 6, "seccomp": {"mode":"filter","no_new_privs":false}, "start_time": "2019-10-20T07:43:13.000Z"}}}
2019-10-20T07:43:13.761Z INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.3.1
2019-10-20T07:43:13.761Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.3.1' as ILM is enabled.
2019-10-20T07:43:13.763Z INFO elasticsearch/client.go:170 Elasticsearch url: http://elasticsearch:9200
2019-10-20T07:43:13.767Z INFO [publisher] pipeline/module.go:97 Beat name: cfb6d8d1e59e
2019-10-20T07:43:13.768Z ERROR instance/beat.go:877 Exiting: Error getting filesets for module zeek: open /usr/share/filebeat/module/zeek: no such file or directory Exiting: Error getting filesets for module zeek: open /usr/share/filebeat/module/zeek: no such file or directory
from docker-zeek.
aaaaah 😧 that's right... Zeek is a x-pack module so NOT oss.
from docker-zeek.
ok, I tested it all myself and it is now working, but kibana takes a LONG time to finish booting now (3-5 mins) so make sure you wait for it to be ready before you start the filebeat container and it should be all good now. 🤞
from docker-zeek.
I can run all containers now. That’s good. However, I see no data.
I have tried breaking it down; I get data from Zeek - when tcpdumping the NIC, I see a lot of traffic and conn.log looks fine. However, I can’t find any data in Kibana.
Also, my ultimate goal is to run this distributed; zeek and filebeat on two sensor boxes and sending data to my Unraid server with Kibana running in Docker. Is that possible? If so, could you please point me to documentation on how to do this? Not nescessarily something that works in Unraid, but just docker-related in general.
Thanks!
from docker-zeek.
Hey, did you see my last comment? Thanks :)
from docker-zeek.
are you running with --cap-add=NET_RAW --net=host
on the zeek container?
from docker-zeek.
Yep. None the less I tried to redo everything and this time there’s a filebeat source somewhere in elasticsearch. I know absolutely nothing about Kibana or Elasticsearch so i guess that’s a good occasion to learn.
Regarding my last comment about distributing the different part of the stack to different Docker servers is that possible?
Thanks
/klaus
from docker-zeek.
I don't quite follow. So it IS working now?
from docker-zeek.
Yeah. I have no idea why it all of a sudden works. I can even find some data in Kibana and visualize something.
Should I create a separate issue for the question regarding running the containers distributed?
Thanks!
/k
from docker-zeek.
klaus, I'm glad it is working now. I believe your follow up question is out of scope for the support I am willing to give for this repo. Docker is it's own thing and Zeek is it's own thing I am merely gluing them together. One day I might write a blog about how to deploy this in a real corp scenario, but for now I am WAY to busy. In the meantime I am sure either the docker or zeek people could help.
thank you and best of luck!
from docker-zeek.
Related Issues (20)
- Path for .zeek files described in the documentation is not loaded by Zeek HOT 1
- Extremely helpful!! Need some assistance HOT 3
- Json output? HOT 10
- Integration question
- add community-id HOT 1
- Docker Compose file with Zeek, Snort, and ELK?
- Error when building zeek:elastic HOT 7
- docker builds for arm HOT 3
- Monitor multi interfaces
- January 11, 2022 Final brownout HOT 2
- suggest for github action
- Forget a newline
- restart: on-failure:5
- Hope to update the package version
- I found that the image didn't seem to work, but it worked when I recompiled it HOT 2
- zeekctl not found
- How to make Zeek container communicate with custom Kafka container
- Zeek container start errors HOT 1
- Do you have plan for zeek 5.0? HOT 1
- Adding Community ID as default HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-zeek.