Giter Club home page Giter Club logo

Comments (13)

blacktop avatar blacktop commented on June 5, 2024

Thank you for pointing this out to me. Yes, I switch all my elastic images over to the OSS versions a while back and forgot to update filebeat. 🤦‍♂️

Thank you!! All images are building in DockerHUB now, so in a few hours it should work. 👍

Please let me know if there is still an issue.

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

Hi again

Seems to have had no effect:

docker run --init --rm -it -v `pwd `:/pcap --link kibana --link elasticsearch blacktop/filebeat:7.4.0 -e
Gives me the exact same error. :-(

Thanks for all your work!

/klaus

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

Apologies, the filebeat image should now be good for 7.4.0

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

Hi again

Unfortunately, it still doesn't work. Integration or configuration towards zeek seem to be missing..

Can you tell me what's wrong? As I understand it, something is missing from the Filebeat Docker image. Correct?

klaus@docker:~$ docker run --init --rm -it -v `pwd`:/pcap --link kibana --link elasticsearch blacktop/filebeat -e
Unable to find image 'blacktop/filebeat:latest' locally
latest: Pulling from blacktop/filebeat
9d48c3bd43c5: Already exists                                                                                                                                                        ef47bfec7715: Pull complete                                                                                                                                                         c25e4b428d67: Pull complete                                                                                                                                                         c3004d8ecbc2: Pull complete                                                                                                                                                         db9be8c78521: Pull complete                                                                                                                                                         4c745d05ed61: Pull complete                                                                                                                                                         Digest: sha256:b48d67331c9cc60a6bec7c901101f90f220dd2ce36b244a545796804c478e760                                                                                                     Status: Downloaded newer image for blacktop/filebeat:latest

===> Waiting on elasticsearch(elasticsearch:9200) to start...
Elasticsearch is ready!
===> Waiting for Kibana(kibana:5601) to start...Kibana is ready!
===> Setting up filebeat...
2019-10-20T07:43:13.746Z        INFO    instance/beat.go:606    Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-10-20T07:43:13.755Z        INFO    instance/beat.go:614    Beat ID: 1746db25-63d0-498f-b50a-2c97ddcaa30a
2019-10-20T07:43:13.756Z        INFO    [beat]  instance/beat.go:902    Beat info       {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "1746db25-63d0-498f-b50a-2c97ddcaa30a"}}}
2019-10-20T07:43:13.756Z        INFO    [beat]  instance/beat.go:911    Build info      {"system_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:11:55.000Z", "version": "7.3.1"}}}
2019-10-20T07:43:13.757Z        INFO    [beat]  instance/beat.go:914    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.12.4"}}}   2019-10-20T07:43:13.758Z        INFO    [beat]  instance/beat.go:918    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-10-17T13:25:15Z","containerized":true,"name":"cfb6d8d1e59e","ip":["127.0.0.1/8","172.17.0.9/16"],"kernel_version":"4.19.0-6-amd64","mac":["02:42:ac:11:00:09"],"os":{"family":"","platform":"alpine","name":"Alpine Linux","version":"","major":0,"minor":0,"patch":0},"timezone":"UTC","timezone_offset_sec":0}}}
2019-10-20T07:43:13.761Z        INFO    [beat]  instance/beat.go:947    Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 12, "ppid": 6, "seccomp": {"mode":"filter","no_new_privs":false}, "start_time": "2019-10-20T07:43:13.000Z"}}}
2019-10-20T07:43:13.761Z        INFO    instance/beat.go:292    Setup Beat: filebeat; Version: 7.3.1
2019-10-20T07:43:13.761Z        INFO    [index-management]      idxmgmt/std.go:178      Set output.elasticsearch.index to 'filebeat-7.3.1' as ILM is enabled.
2019-10-20T07:43:13.763Z        INFO    elasticsearch/client.go:170     Elasticsearch url: http://elasticsearch:9200
2019-10-20T07:43:13.767Z        INFO    [publisher]     pipeline/module.go:97   Beat name: cfb6d8d1e59e
2019-10-20T07:43:13.768Z        ERROR   instance/beat.go:877    Exiting: Error getting filesets for module zeek: open /usr/share/filebeat/module/zeek: no such file or directory    Exiting: Error getting filesets for module zeek: open /usr/share/filebeat/module/zeek: no such file or directory

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

aaaaah 😧 that's right... Zeek is a x-pack module so NOT oss.

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

ok, I tested it all myself and it is now working, but kibana takes a LONG time to finish booting now (3-5 mins) so make sure you wait for it to be ready before you start the filebeat container and it should be all good now. 🤞

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

I can run all containers now. That’s good. However, I see no data.

I have tried breaking it down; I get data from Zeek - when tcpdumping the NIC, I see a lot of traffic and conn.log looks fine. However, I can’t find any data in Kibana.

Also, my ultimate goal is to run this distributed; zeek and filebeat on two sensor boxes and sending data to my Unraid server with Kibana running in Docker. Is that possible? If so, could you please point me to documentation on how to do this? Not nescessarily something that works in Unraid, but just docker-related in general.

Thanks!

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

Hey, did you see my last comment? Thanks :)

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

are you running with --cap-add=NET_RAW --net=host on the zeek container?

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

Yep. None the less I tried to redo everything and this time there’s a filebeat source somewhere in elasticsearch. I know absolutely nothing about Kibana or Elasticsearch so i guess that’s a good occasion to learn.

Regarding my last comment about distributing the different part of the stack to different Docker servers is that possible?

Thanks

/klaus

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

I don't quite follow. So it IS working now?

from docker-zeek.

klausagnoletti avatar klausagnoletti commented on June 5, 2024

Yeah. I have no idea why it all of a sudden works. I can even find some data in Kibana and visualize something.

Should I create a separate issue for the question regarding running the containers distributed?

Thanks!

/k

from docker-zeek.

blacktop avatar blacktop commented on June 5, 2024

klaus, I'm glad it is working now. I believe your follow up question is out of scope for the support I am willing to give for this repo. Docker is it's own thing and Zeek is it's own thing I am merely gluing them together. One day I might write a blog about how to deploy this in a real corp scenario, but for now I am WAY to busy. In the meantime I am sure either the docker or zeek people could help.

thank you and best of luck!

from docker-zeek.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.