Giter Club home page Giter Club logo

graylog-fortigate-cef's Introduction

Fortigate CEF Logs - Graylog Content Pack

Notice

This project has been depricated. Please use https://github.com/seanthegeek/graylog-fortigate-syslog instead.

This Graylog content pack includes a steam and dashboards for Fortinet Fortigate Common Event Format (CEF) logs.

Streams

Fortigate CEF Logs

Routes CEF logs from Fortigates to the Fortigate CEF Logs Graylog index set

Dashboards

Fortigate - Applications and Devices

Analysis of devices and application traffic

Includes IP addresses, MAC addresses, device manufacturers, and application layer network traffic

Fortigate - DNS Traffic

Details of DNS queries and responses

Includes details of the query, response, action, and category

Fortigate - IPS Alerts

Intrusion Prevention System (IPS) alert details

Includes signature, action, severity, source, and destination information

Fortigate - Overview

An overview of incoming messages from Fortigates

Includes Fortigate hostnames, serial numbers, and full message details

Fortigate - SSL/TLS Interventions

SSL/TLS actions taken by Fortigates

Provides records of when Fortigates intervened (with or without decrypting) in SSL/TLS traffic

Fortigate - Web Traffic

Web traffic details

Includes category, action, and more

Searches

Fortigate CEF

All Fortigate CEF logs

Graylog Setup

Edit the Graylog server configuration file at /etc/graylog/server/server.conf. Locate the allow_leading_wildcard_searches and allow_highlighting options, and set both to true. Restart the Graylog server by running sudo systemctl restart graylog-server.service.

Import the Content Pack into Graylog by navigating to System> Content Packs, clicking on the upload button, and uploading the Content Pack JSON file.

In Graylog an Input accepts log traffic from a source an parses it. That data is sent to Streams, which filters and routes log traffic to Index Sets. Index Sets manage the Elasticsearch indexes that Graylog uses as a backend.

The Stream that comes with this content pack is configured to route the logs to a separate Index Set called Fortigate CEF Logs. It does not create the Index Set, so the Index Set needs to be created.

Navigate to System> Indices, and create a new Index Set with a title of Fortigate CEF Logs and an index prefix of fortigate_cef. Then, click on Streams in the main navigation bar. Edit the Fortigate CEF Logs Stream and ensure it is configured to use the Index Set that you just created.

Important: Leave Remove matches from ‘All messages’ stream box checked, or the data will be duplicated over two Index Sets.

Create a CEF UDP or a CEF TCP input by navigating to System> Inputs as a Graylog administrator, and clicking on Launch New Input.

Before creating a CEF TCP input:

Ensure that your certificate and and key are readable by the user running Graylog, or Graylog will create it's own self-signed certificate (which Fortigates will not trust) without informing you in the web UI (this error can be found in server.log.)

It is recommended to use a commercial external Certificate Authority (CA). Documentation contributions for using internal CAs would be appreciated. Documentation for using Let's Encrypt Certificates is in progress.

When creating a CEF TLS Input, be sure to check the Accept encrypted connections checkbox.

Fortigate setup

Configure your Fortigates to send data to Graylog in CEF format by using the FortiOS Command Line Interface (CLI).

Replace the server address and port with the address and port of your input, of course.

Time zone

To simplify and unify log management, it is important that every firewall be configured to use the GMT timezone, which for logging purposes is equivalent UTC.

config system global
    set timezone 80
end

Log filtering

By default, logs sent to the syslog server are not filtered. To ensure that the Graylog Input gets all logs, ensure all log filter options are at their default settings.

config log syslogd filter
    unset severity
    unset forward-traffic
    unset local-traffic
    unset multicast-traffic
    unset sniffer-traffic
    unset anomaly
    unset voip
end

CEF UDP

Warning : UDP traffic is unencrypted.

config log syslogd setting
    set status enable
    set server "graylog.example.com"
    set port 5555
    set format cef
    set mode udp
end

CEF TCP

Warning: When using CEF TCP, the 'server' setting must be set the Graylog server's fully-qualified hostname, not the IP address.

config log syslogd setting
    set status enable
    set server "graylog.example.com"
    set port 5555
    set format cef
    set mode reliable
end

graylog-fortigate-cef's People

Contributors

seanthegeek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

graylog-fortigate-cef's Issues

Question on Fortigate needing hostname for Graylog

Hi there!

This is a great contribution for Fortigate/Graylog users. May I ask why the Fortigate needs the syslog server's hostname vs. an IP? The README's example is not using certs, so I am failing to understand how the hostname is qualified other than a DNS lookup.

Thanks!

Missing Fields/Extractors

I installed your content pack but the dashboards are empty. I found out that the fields are missing. Do you have the extractor to import?

Installing content pack failed with status: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Failed to install content pack <af84f707-7473-4258-bb2a-9d9617247bdd/16>. Could not install Content Pack with ID: af84f707-7473-4258-bb2a-9d9617247bdd

Hello!
I can not install your pack. My graylog version 4.2.9 and have followed Readme of your pack. But get this error

Installing content pack failed with status: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Failed to install content pack <af84f707-7473-4258-bb2a-9d9617247bdd/16>. Could not install Content Pack with ID: af84f707-7473-4258-bb2a-9d9617247bdd

Also attach my server.log. Please can ypu help me, I can not figure out what is problem?

Caused by: org.graylog2.contentpacks.exceptions.ContentPackException: Missing Stream for widget entity
>at org.graylog2.contentpacks.model.entities.WidgetEntity.lambda$toNativeEntity$1(WidgetEntity.java:151) ~[graylog.jar:?]
>at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_312]
>at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_312]
>at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_312]
>at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1580) ~[?:1.8.0_312]
>at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_312]
>at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_312]
>at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_312]
>at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_312]
>at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566) ~[?:1.8.0_312]
>at org.graylog2.contentpacks.model.entities.WidgetEntity.toNativeEntity(WidgetEntity.java:159) ~[graylog.jar:?]
>at org.graylog2.contentpacks.model.entities.ViewStateEntity.lambda$toNativeEntity$0(ViewStateEntity.java:125) ~[graylog.jar:?]
>at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_312]

Failed installed content pack

Hi, I have tried to install the content pack with the below error.
Installing content pack failed with status: FetchError: There was an error fetching a resource: Internal Server Error. Additional information: Failed to install content pack <af84f707-7473-4258-bb2a-9d9617247bdd/16>. Could not install Content Pack with ID: af84f707-7473-4258-bb2a-9d9617247bdd

CEF TCP / Unable to decode raw message

When I configure the FortiGate to send CEF logs over TCP I get errors like below:

2022-01-21T09:14:06.052Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=7b981742-7a9a-11ec-a7be-fa163eb46c72, messageQueueId=10158, codec=CEF, payloadSize=1194, timestamp=2022-01-21T09:14:05.876Z, remoteAddress=/**.**.**.**:15394} on input <61ea78ce6455d64533205f9a>. 2022-01-21T09:14:06.052Z ERROR [DecodingProcessor] Error processing message RawMessage{id=7b981742-7a9a-11ec-a7be-fa163eb46c72, messageQueueId=10158, codec=CEF, payloadSize=1194, timestamp=2022-01-21T09:14:05.876Z, remoteAddress=/**.**.**.**:15394} java.lang.IllegalStateException: Could not parse timestamp. '1190 <189>Jan 21 09:14:05' at com.github.jcustenborder.cef.CEFParserImpl.parse(CEFParserImpl.java:162) ~[graylog.jar:?] at org.graylog.plugins.cef.codec.CEFCodec.decodeCEF(CEFCodec.java:128) ~[graylog.jar:?] at org.graylog.plugins.cef.codec.CEFCodec.decode(CEFCodec.java:117) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:153) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:94) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:95) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:49) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:829) [?:?]

Input:
bind_address: 0.0.0.0 locale: <empty> max_message_size: 2097152 number_worker_threads: 2 port: 5555 recv_buffer_size: 1048576 tcp_keepalive: false timezone: Etc/UTC tls_cert_file: /etc/letsencrypt/live/***/fullchain.pem tls_client_auth: disabled tls_client_auth_cert_file: <empty> tls_enable: false tls_key_file: /etc/letsencrypt/live/***/privkey.pem tls_key_password:******** use_full_names: false use_null_delimiter: true

Do you have any idea how to solve it ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.