graylog2 / graylog-plugin-collector Goto Github PK

Collector plugin for Graylog

License: Other

JavaScript 52.77% Java 46.57% CSS 0.30% Groovy 0.02% TypeScript 0.35%

graylog graylog-plugin collector-sidecar filebeat winlogbeat nxlog

graylog-plugin-collector's Introduction

Collector Plugin for Graylog

Graylog Collector is a lightweight Java application that allows you to forward data from log files to a Graylog cluster. The collector can read local log files and also Windows Events natively, it then can forward the log messages over the network using the GELF format.

⚠️ The Graylog Collector Sidecar is deprecated and can be replaced with Graylog Sidecar

Required Graylog version: 2.0 and later

Installation

Download the plugin and place the .jar file in your Graylog plugin directory. The plugin directory is the plugins/ folder relative from your graylog-server directory by default and can be configured in your graylog.conf file.

Restart graylog-server and you are done.

Development

You can improve your development experience for the web interface part of your plugin dramatically by making use of hot reloading. To do this, do the following:

git clone https://github.com/Graylog2/graylog2-server.git
cd graylog2-server/graylog2-web-interface
ln -s $YOURPLUGIN plugin/
npm install && npm start

Usage

Please refer to the Documentation.

Getting started

This project is using Maven 3 and requires Java 7 or higher.

Clone this repository.
Run mvn package to build a JAR file.
Optional: Run mvn jdeb:jdeb and mvn rpm:rpm to create a DEB and RPM package respectively.
Copy generated JAR file in target directory to your Graylog plugin directory.
Restart the Graylog.

Plugin Release

We are using the maven release plugin:

$ mvn release:prepare
[...]
$ mvn release:perform

This sets the version numbers, creates a tag and pushes to GitHub. Travis CI will build the release artifacts and upload to GitHub automatically.

graylog-plugin-collector's People

Contributors

Stargazers

Watchers

Forkers

gitfrederic rlugojr hamphh73 bingzijian kira8565 npdtools-verizon silenceper d-demirci mysky528 barrysofast drsantos20 tulethanh benhe119 sonvt1710 isabella232 ekmixon uskr lvhaiyan888

graylog-plugin-collector's Issues

Remove RestBoolean from CollectorConfigurationResource

The class RestBoolean seems no longer necessary, as the corresponding Javascript store always passes true now.
It just needlessly complicates the intended resource, making it harder to follow, boolean flags should simply use boolean datatypes.

nxlog: Explicitly set ShortMessageLength

The nxlog GELF output by default cuts off messages (short_message) at 64 characters which might be unintuitive to most users.

The ShortMessageLength setting for nxlog should explicitly be set to a higher (or configurable value) to avoid incomplete messages.

From https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_gelf

ShortMessageLength
This optional directive can be used to specify the length of the short_message field. This defaults to 64 if the directive is not explicitly specified. If the field short_message or ShortMessage is present, it will not be truncated.

https://nxlog.co/question/1037/short-messages-truncated-64-chars

Optional input field to manage a complete collector configuration file

For people who prefer to manage the complete configuration of a collector instead of using the input boxes that will generate one, it would be great to be able to just have a text area field that takes a configuration, which will then be used on the client machine.

Wrong URL in the submenu

When we run graylog with the URL 127.0.0.1:9000/graylog the submenu "Collectors" is linked to 127.0.0.1:9000/system/collectors instead of 127.0.0.1:9000/graylog/system/collectors .

We can access to the page by manually browsing to 127.0.0.1:9000/graylog/system/collectors

Thanks

Think on how to do "show messages" functionality with new config

Setting a collector ID is no longer under our control, so it depends on users' configuration. We need to discuss if we want to keep this functionality (and how) or delete it.

Mark a Sidecar with status `unknown` when there was no update

Sidecars are removed/marked as inactive after some time if there was no status update. It would be nice to first mark it with the status unknown (no update from the node for a short time) and then after a second period delete/inactivate the node.

Move into graylog2-server

Check if anything is using the fully qualified class names before changing the packages. We might need migrations to fix this.

Examples:

Cluster config
Other MongoDB database objects
Config file settings.

Also move all open issues to the new repo.

Add a view of the generated config for a collector backend

Add a view of the generated config file under the collector status page so it can be easily viewed and verified centrally. Maybe a 'view config' button next to the current restart button?

Newline is broken on Windows when using Beats snippets

Creating a Beats snippet which extends on multiple lines does not work on Windows.
For example, the following snippets:

winlogbeat:
  event_logs:
  - name: Application
    ignoreolder: 168h
  - name: System
  - name: Security

produces (on Windows) the following one-liner:

winlogbeat: event_logs: - name: Application ignoreolder: 168h - name: System - name: Security

which clearly breaks the Windows client.

Checking the source code, I think the problematic entry is here: it should use the platform-specific newline escape rather than hardcoding \n

Update interval configuration problem

When using PT60S as update interval, the sidecar gets the correct value of 60 seconds. When using PT1M, the sidecar gets a wrong value of 0 seconds.

Add button to manage single sidecar in administration page

As we designed in mockups.

Implement administration actions

Add process actions i.e. start, stop, restart.

Display collector IP address

Problem description

As an admin of Graylog environments where installation and configuration of the Sidecar is delegated to various teams, I would like the ability to view a collector's IP address alongside the other collector status information. This would be useful in cases where a collector may not have an accurate name (e.g. "localhost") and/or I need to assist with troubleshooting an issue.

I have set up a logger for org.graylog2.rest.accesslog, so I can dig in to get an IP if absolutely needed, but it would be much more convenient to have that info displayed with the other collector info.

Environment

Graylog Version: 2.0.3
Elasticsearch Version: 2.2.0
MongoDB Version: 2.6.12
Operating System: Red Hat Enterprise Linux Server release 6.7
Browser version: Firefox 47

include support for symlinks

Is it possible to include support for symlinks at graylog-plugin-collector/src/web/collector-configuration/EditInputFields.jsx?

Or is it any workaround to get symlinks enabled?

For reference, this is the issue I'm facing https://discuss.elastic.co/t/how-to-enable-symlinks/126852

Thx!

Review administration page

Gather feedback about UI
Should we only display active sidecars?
How to treat filtering of inactive sidecars?

Delay in processing and displaying results on Graylog web interface using Filebeat

Problem description

The graylog web interface is displaying messages from the filebeat input almost 15 minutes after being processed. I don't have this problem with the syslog udp input. I have checked the sever's, user's and web browser time configurations and they are all the same. The elasticsearch server is separate but I'm using an NTP server to sync the time.

Config

collector_sidecar.yml:
server_url: http://graylogserver_ip:9000/api/
update_interval: 30
tls_skip_verify: true
send_status: true
list_log_files:
- /var/log
node_id: graylog-collector-sidecar-nodename
collector_id: file:/etc/graylog/collector-sidecar/collector-id
cache_path: /var/cache/graylog/collector-sidecar
log_path: /var/log/graylog/collector-sidecar
log_rotation_time: 86400
log_max_age: 604800
tags:
- syslog
- linux
backends:
- name: nxlog
enabled: false
binary_path: /usr/bin/nxlog
configuration_path: /etc/graylog/collector-sidecar/generated/nxlog.conf
- name: filebeat
enabled: true
binary_path: /usr/bin/filebeat
configuration_path: /etc/graylog/collector-sidecar/generated/filebeat.yml

Environment

filebeat version 5.1.1 (amd64), libbeat 5.1.1
Graylog Collector Sidecar version 0.1.0 (e14968e) [go1.7.4/amd64]
Graylog-2.2

Add support for content packs

Allow users to export configuration and log collectors through content packs.

Configuration caching issue

I started with a new Graylog installation, empty database and thus without any collector configuration.

Apart from #36, I am also seeing that the filebeat and nxlog configurations are not updated once I created a configuration in the server. Only after restarting the sidecar process the new configurations get fetched.

I guess there is an issue with the new configuration caching system. To reproduce it, it should be enough to start a sidecar with a tag for which no configuration exists.

This is a multi node setup of Graylog 2.2.0-alpha.5-SNAPSHOT+1720c48 with sidecar 5e577167b55db1bdd4d99c05cacfe0d2fe13593a.

Check for possible performance improvements

This issue will be used to collect notes about code areas where we should check for possible performance improvements before releasing 2.2.0.

Cache collector system configuration for registration response in CollectorResource

Ability to clone collector configuration

Problem description

I'd like to request a mechanism to clone a collector configuration (outputs, inputs, and snippets; potentially excluding tags to avoid Sidecar detection).

We define configurations on a per team, environment, and server type basis. Within a team's "server type" there are only a few minor differences... most notably each environment's unique Graylog input port. It would save time to be able to clone an existing collector configuration and make the 1-2 required changes.

Environment

Graylog Version: 2.0.3
Elasticsearch Version: 2.2.0
MongoDB Version: 2.6.12
Operating System: Red Hat Enterprise Linux Server release 6.7
Browser version: Firefox 47

Adapt sidecar details page to latest changes

The sidecar details page should be adapted and reviewed.

Beats TLS Output Configuration Creating Unnecessary Winlog Config

Checking "Enable TLS Support" in the Winlogbeat Output causes it to force the following in the TLS section of the winlogbeat.yml auto-configuration:
certificate_authorities:
- ""

This causes winlogbeat to fail to execute. The same configuration in Filebeat Output does not cause the bad certificate_authorities field to appear, and that configuration is functional.

Mark old collector plugin as deprecated

This plugin can be used in 3.0 to help with the migration to the new sidecar implementation. To make it clear that it is deprecated, we should adjust the UI.

Change system menu entry from "Collectors" to "Collectors (legacy)" or "Collectors (deprecated)"
Add a deprecation notification to all legacy collector pages

Plugin availability as a jar?

The plugin appears to be available source only.
The instructions imply availability as a jar (pre-built) for deploy.
Is the plugin available as a pre-built jar?
Thanks,

Rob

Plugin installation

Where could I find the .jar file?

Support new collectors

Topbeat (meh, but quite a lot of customers asked for it)
Auditbeat (crucial for many security use-cases)
Packetbeat (crucial for many security use-cases)

Create configuration page for collectors and ditch modal

The configuration modal of a collector became quite large over time. An extra configuration page for each collector would improve usability. Additionally more space would allow to provide more help on each configuration option.

Beats Collector Output "Type" values

Expected Behavior

The Type column of the Beats Outputs table shows something indicating [FileBeat] Beats output or [WinLogBeat] Beats output, or the Create/Edit Output form's Type select values indicate logstash.

Current Behavior

The Type column of the Beats Outputs table shows logstash while the Type select on the Create/Edit Output form shows either [FileBeat] Beats output or [WinLogBeat] Beats output.

Steps to Reproduce (for bugs)

Get a new 2.1.0 install
Log in as an admin and navigate to System --> Collectors --> Manage configurations
Create a new configuration (or select and existing one) and click its Name link
Ensure the Beats sub-tab is selected and click Create Output
Create and save an output with a Type of either [FileBeat] Beats output or [WinLogBeat] Beats output
Note the Type column shows logstash

Context

logstash wasn't mentioned on the Create/Edit form, so it was a bit odd to see it in the Beats Output table. I assume this is just multiple meanings of Type (protocol + output type).

Environment

Graylog Version: 2.1.0+62db7e0
Elasticsearch Version: 2.2.0
MongoDB Version: 2.6.12
Operating System: Red Hat Enterprise Linux Server release 6.7
Browser version: Firefox 48.0.2

Download pre-configured sidecar from graylog-server

It would be great to be able to download the sidecar binary directly from a graylog-server process through the REST API and have it ship a configuration file that already points to the right graylog-server for the first check in.

Incomplete input specification makes CollectorConfiguration throw exception on render

See Graylog2/graylog2-server#2247.

Move collector logic to graylog-server

Currently, to support a new collector, logic has to be added in the graylog-server plugin and the collector-sidecar binary.

This obviously adds complexity in development and version compatibility (including enforced upgrades on all your machines that run the sidecar) but also requires golang knowledge.

Is there a way we could make the collector-sidecar dumber and the graylog-server plugin do more? Maybe we could have the collector-sidecar purely manage configuration files and processes but nothing else, while the graylog-server plugin ships complete configuration files?

Issues #69, #70 might be very much related to this.

Sidecar nxlog input/output view

I've found two issues in Beta 1 of 2.1 (I have not put beta 2 in so apologies if this was already addressed but didn't see it). One I think is a bug, one is... an inconvenient feature perhaps.

The first I'd call a bug is that the text entered in the "arbitrary text" is inserted without a new line following it. Consider the "input" section. If you put a line of code, without adding an empty line, the resulting code in the config file is:

Exec If $SourceName == 'Microsoft-Windows-StorDiag' drop();</Input>

The trailing close of the input tag is the a syntax error. I can't think of any circumstance where you want code with the closing tag appended before line end, so you will cut down on a lot of (fairly hidden) errors if you ensure the arbitrary text ends in a new line.

The other issue is that the section is hidden (initially collapsed). I get that you did that to reduce casual use and mistakes, but once it has content, it should always be shown expanded. Otherwise if you go into the dialog, it is very easy to get confused and think there is no text there. I would suggest that if you want it initially collapsed, only do so when it is empty.

Thanks,
Linwood

why do not add "exclude_files" properties for filebeat collect ?

From @silenceper on September 14, 2017 8:22

I use filebeat to collect message, but just specified the file path is too simple for me.

so i want to use the "exclude_files" properties.

Copied from original issue: Graylog2/graylog2-server#4145

Versioned configurations

It would be great to have versioned configurations that you can roll back to in case you made an error in a new configuration. It would also make debugging easier by seeing when a change happened and what change that was.

The sidecar makes changing settings so easy (that's its goal!) that we need to make making accidental or erroneous changes as hard as possible.

better log files and error handling

If you start Sidecar with a doubled node-id it will not fetch any configuration and writes to logfile that it is not able to fetch the configuration:

Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="collector-id file doesn't exist, generating a new one"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Using collector-id: 598f10fd-10cc-4d92-a9f1-c7960d2bb4b1"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Fetching configurations tagged by: [linux honeypot mysql]"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Starting signal distributor"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="collector-id file doesn't exist, generating a new one"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Using collector-id: 598f10fd-10cc-4d92-a9f1-c7960d2bb4b1"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Fetching configurations tagged by: [linux honeypot mysql]"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Starting signal distributor"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 1/3"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 1/3"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 2/3"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 2/3"
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Configuration change detected, rewriting configuration file."
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Configuration change detected, rewriting configuration file."
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Stopping"
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Stopping"
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:52 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:52 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 3/3"
Oct 18 16:06:52 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:52 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 3/3"
Oct 18 16:06:57 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:57+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:57 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:57+02:00" level=info msg="[filebeat] Starting (exec driver)"

The Server Logfile did not give any Error. User should get the message that the creation of this Sidecar and the register was not successfull.

Form states not reset when switching between Beats and NXLog

Problem description

The form state is not reset when switching between Beats and NXLog on the Collector Configuration page (/system/collectors/configurations/ID). This is definitely minor... I only noticed it when I realized I was in the Beats section rather than NXLog.

Steps to reproduce the problem

Log in as an admin
Navigate to /system/collectors/configurations/ and create a new collector configuration
Click the link for the new configuration
With the Beats sub-tab select, click "Create Output"
Enter "abc" as the name and "[FileBeat] Beats output" as the type
Click the "Cancel" button (NOTE: "Save" does reset the form state as expected)
Select the "NXLog" sub-tab and click "Create Output"
Note the Name is "abc" and Type is "filebeat:logstash"
Repeat with Inputs and Snippets

Environment

Graylog Version: 2.1.0-beta.1+1b940e4
Elasticsearch Version: 2.2.0
MongoDB Version: 2.6.12
Operating System: Red Hat Enterprise Linux Server release 6.7
Browser version: Firefox 47

Select initial collector sub-tab based upon whether existing config is defined

Expected Behavior

If a collector configuration has outputs/inputs/snippets for one collector type (e.g. NXLog) but not any other types (e.g. Beats) the sub-tab for that populated type should be pre-selected when the page loads.

Current Behavior

The "Beats" sub-tab is always selected first.

Steps to Reproduce (for bugs)

Create a collector configuration with some NXLog, but no Beats, configuration
Navigate to some other Graylog page
Navigate back to the collector configuration's page
Note the "Beats" sub-tab is selected, even though you only have configuration for NXLog

Context

This is minor, but a bit annoying as we (currently) don't use Beats.

Your Environment

Graylog Version: 2.1.1+01d50e5
Elasticsearch Version: 2.2.0
MongoDB Version: 2.6.12
Operating System: Red Hat Enterprise Linux Server release 6.8
Browser version: Firefox 49.0.1

NXLog GELF/TLS input should work without cert files

The NXLog GELF TLS output should work without any cert files given.

System/Collectors not available

I'm running Graylog v2.2.2 inside a docker container. I have two plugins running, beats input and collector. Both plugins show up on the configuration page.

I have a sidecar running on the physical host. However the option to configure collector sidecar presumably under http://ip:9000/system/collectors does not exist. Is this a configuration mistake somewhere? Do I need to enable this plugin with additional options?

Dependabot couldn't authenticate with https://oss.sonatype.org/content/repositories/releases

Dependabot couldn't authenticate with https://oss.sonatype.org/content/repositories/releases.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

You can mention @dependabot in the comments below to contact the Dependabot team.

Backwards incompatibility with new collector system config

The new CollectorSystemConfiguration values need to be @Nullable to ensure backwards compatibility.

2016-10-17 13:30:55,511 ERROR: org.graylog2.cluster.ClusterConfigServiceImpl - Error while deserializing payload
java.lang.IllegalArgumentException: Can not construct instance of org.graylog.plugins.collector.system.CollectorSystemConfiguration, problem: Missing required properties: collectorUpdateInterval
 at [Source: N/A; line: -1, column: -1]
    at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3580) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3499) ~[jackson-databind-2.8.3.jar:2.8.3]
    at org.graylog2.cluster.ClusterConfigServiceImpl.extractPayload(ClusterConfigServiceImpl.java:94) [classes/:?]
    at org.graylog2.cluster.ClusterConfigServiceImpl.get(ClusterConfigServiceImpl.java:110) [classes/:?]
    at org.graylog2.cluster.ClusterConfigServiceImpl.get(ClusterConfigServiceImpl.java:120) [classes/:?]
    at org.graylog2.cluster.ClusterConfigServiceImpl.getOrDefault(ClusterConfigServiceImpl.java:125) [classes/:?]
    at org.graylog.plugins.collector.system.CollectorSystemConfigurationSupplier.get(CollectorSystemConfigurationSupplier.java:45) [classes/:?]
    at org.graylog.plugins.collector.system.CollectorSystemConfigurationSupplier.get(CollectorSystemConfigurationSupplier.java:25) [classes/:?]
    at org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread.doRun(PurgeExpiredCollectorsThread.java:83) [classes/:?]
    at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:77) [classes/:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_101]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_101]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_101]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_101]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of org.graylog.plugins.collector.system.CollectorSystemConfiguration, problem: Missing required properties: collectorUpdateInterval
 at [Source: N/A; line: -1, column: -1]
    at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:268) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1441) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:468) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:487) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:276) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:224) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:471) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1196) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3575) ~[jackson-databind-2.8.3.jar:2.8.3]
    ... 16 more
Caused by: java.lang.IllegalStateException: Missing required properties: collectorUpdateInterval
    at org.graylog.plugins.collector.system.AutoValue_CollectorSystemConfiguration$Builder.build(AutoValue_CollectorSystemConfiguration.java:144) ~[classes/:?]
    at org.graylog.plugins.collector.system.CollectorSystemConfiguration.create(CollectorSystemConfiguration.java:56) ~[classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
    at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:117) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:274) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:224) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:471) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1196) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148) ~[jackson-databind-2.8.3.jar:2.8.3]
    at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3575) ~[jackson-databind-2.8.3.jar:2.8.3]
    ... 16 more
2016-10-17 13:30:55,517 ERROR: org.graylog2.cluster.ClusterConfigServiceImpl - Couldn't extract payload from cluster config (type: org.graylog.plugins.collector.system.CollectorSystemConfiguration)

NullPointerException when collector config is empty

When starting with a fresh MongoDB database, the collector configuration resource throws a NullPointerException.

2016-11-09 10:36:26,897 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
java.lang.NullPointerException: null
        at org.graylog.plugins.collector.configurations.rest.resources.CollectorConfigurationResource.getConfiguration(CollectorConfigurationResource.java:110)
 ~[?:?]
        at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_111]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_111]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog
.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?
]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:
?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.j
ava:160) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar
:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_111]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_111]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

mvn package failed

mvn package

file：graylog-plugin-collector/target/surefire-reports
content:

-------------------------------------------------------------------------------
Test set: org.graylog.plugins.collector.collectors.CollectorServiceImplTest
-------------------------------------------------------------------------------
Tests run: 10, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 6.299 sec <<< FAILURE! - in org.graylog.plugins.collector.collectors.CollectorServiceImplTest
testFindByNodeId(org.graylog.plugins.collector.collectors.CollectorServiceImplTest)  Time elapsed: 0.564 sec  <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

 at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)
Caused by: java.lang.Error:
Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)

testAll(org.graylog.plugins.collector.collectors.CollectorServiceImplTest)  Time elapsed: 0.044 sec  <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

 at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)
Caused by: java.lang.Error:
Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)

testFindById(org.graylog.plugins.collector.collectors.CollectorServiceImplTest)  Time elapsed: 0.025 sec  <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

 at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)
Caused by: java.lang.Error:
Unresolved compilation problem:
	AutoValue_CollectorNodeDetails cannot be resolved to a type

	at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)

testSaveFirstRecord(org.graylog.plugins.collector.collectors.CollectorServiceImplTest)  Time elapsed: 0.024 sec  <<< ERROR!
java.lang.Error:
Unresolved compilation problem:

........

Adapt to changes in PropTypes

PropTypes have moved into a separate package (prop-types), so all imports and usages of the old way (being part of the React module) are changed.

For more details see
https://facebook.github.io/react/blog/2017/04/07/react-v15.5.0.html#migrating-from-react.proptypes

Refs Graylog2/graylog2-server#4059

graylog2 / graylog-plugin-collector Goto Github PK

graylog-plugin-collector's Introduction

Collector Plugin for Graylog

Installation

Development

Usage

Getting started

Plugin Release

graylog-plugin-collector's People

Contributors

Stargazers

Watchers

Forkers

graylog-plugin-collector's Issues

Problem description

Environment

Problem description

Config

Environment

Problem description

Environment

Expected Behavior

Current Behavior

Steps to Reproduce (for bugs)

Context

Environment

Problem description

Steps to reproduce the problem

Environment

Expected Behavior

Current Behavior

Steps to Reproduce (for bugs)

Context

Your Environment

Recommend Projects

Recommend Topics

Recommend Org