graylog2 / graylog-plugin-collector Goto Github PK
View Code? Open in Web Editor NEWCollector plugin for Graylog
License: Other
Collector plugin for Graylog
License: Other
mvn package
file:graylog-plugin-collector/target/surefire-reports
content:
-------------------------------------------------------------------------------
Test set: org.graylog.plugins.collector.collectors.CollectorServiceImplTest
-------------------------------------------------------------------------------
Tests run: 10, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 6.299 sec <<< FAILURE! - in org.graylog.plugins.collector.collectors.CollectorServiceImplTest
testFindByNodeId(org.graylog.plugins.collector.collectors.CollectorServiceImplTest) Time elapsed: 0.564 sec <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)
Caused by: java.lang.Error:
Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindByNodeId(CollectorServiceImplTest.java:140)
testAll(org.graylog.plugins.collector.collectors.CollectorServiceImplTest) Time elapsed: 0.044 sec <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)
Caused by: java.lang.Error:
Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testAll(CollectorServiceImplTest.java:99)
testFindById(org.graylog.plugins.collector.collectors.CollectorServiceImplTest) Time elapsed: 0.025 sec <<< ERROR!
org.mongojack.MongoJsonMappingException: Error mapping BSON to POJOs
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)
Caused by: com.fasterxml.jackson.databind.JsonMappingException:
Can not construct instance of org.graylog.plugins.collector.collectors.CollectorNodeDetails, problem: Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at [Source: N/A; line: -1, column: -1] (through reference chain: org.graylog.plugins.collector.collectors.CollectorImpl["node_details"])
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)
Caused by: java.lang.Error:
Unresolved compilation problem:
AutoValue_CollectorNodeDetails cannot be resolved to a type
at org.graylog.plugins.collector.collectors.CollectorServiceImplTest.testFindById(CollectorServiceImplTest.java:119)
testSaveFirstRecord(org.graylog.plugins.collector.collectors.CollectorServiceImplTest) Time elapsed: 0.024 sec <<< ERROR!
java.lang.Error:
Unresolved compilation problem:
........
For people who prefer to manage the complete configuration of a collector instead of using the input boxes that will generate one, it would be great to be able to just have a text area field that takes a configuration, which will then be used on the client machine.
This issue will be used to collect notes about code areas where we should check for possible performance improvements before releasing 2.2.0.
CollectorResource
This plugin can be used in 3.0 to help with the migration to the new sidecar implementation. To make it clear that it is deprecated, we should adjust the UI.
Hi
When we run graylog with the URL 127.0.0.1:9000/graylog the submenu "Collectors" is linked to 127.0.0.1:9000/system/collectors instead of 127.0.0.1:9000/graylog/system/collectors .
We can access to the page by manually browsing to 127.0.0.1:9000/graylog/system/collectors
Thanks
Checking "Enable TLS Support" in the Winlogbeat Output causes it to force the following in the TLS section of the winlogbeat.yml auto-configuration:
certificate_authorities:
- ""
This causes winlogbeat to fail to execute. The same configuration in Filebeat Output does not cause the bad certificate_authorities field to appear, and that configuration is functional.
Currently, to support a new collector, logic has to be added in the graylog-server
plugin and the collector-sidecar
binary.
This obviously adds complexity in development and version compatibility (including enforced upgrades on all your machines that run the sidecar) but also requires golang knowledge.
Is there a way we could make the collector-sidecar
dumber and the graylog-server
plugin do more? Maybe we could have the collector-sidecar
purely manage configuration files and processes but nothing else, while the graylog-server
plugin ships complete configuration files?
PropTypes have moved into a separate package (prop-types), so all imports and usages of the old way (being part of the React module) are changed.
For more details see
https://facebook.github.io/react/blog/2017/04/07/react-v15.5.0.html#migrating-from-react.proptypes
I'd like to request a mechanism to clone a collector configuration (outputs, inputs, and snippets; potentially excluding tags to avoid Sidecar detection).
We define configurations on a per team, environment, and server type basis. Within a team's "server type" there are only a few minor differences... most notably each environment's unique Graylog input port
. It would save time to be able to clone an existing collector configuration and make the 1-2 required changes.
It would be great to have versioned configurations that you can roll back to in case you made an error in a new configuration. It would also make debugging easier by seeing when a change happened and what change that was.
The sidecar makes changing settings so easy (that's its goal!) that we need to make making accidental or erroneous changes as hard as possible.
If you start Sidecar with a doubled node-id it will not fetch any configuration and writes to logfile that it is not able to fetch the configuration:
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="collector-id file doesn't exist, generating a new one"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Using collector-id: 598f10fd-10cc-4d92-a9f1-c7960d2bb4b1"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Fetching configurations tagged by: [linux honeypot mysql]"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="Starting signal distributor"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="collector-id file doesn't exist, generating a new one"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Using collector-id: 598f10fd-10cc-4d92-a9f1-c7960d2bb4b1"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Fetching configurations tagged by: [linux honeypot mysql]"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="Starting signal distributor"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:40 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 1/3"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:40 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:40+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 1/3"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:45 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 2/3"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:45 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:45+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 2/3"
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Configuration change detected, rewriting configuration file."
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Configuration change detected, rewriting configuration file."
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Stopping"
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Stopping"
Oct 18 16:06:50 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:50 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:50+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:52 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:52 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 3/3"
Oct 18 16:06:52 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Collector exits immediately, this should not happen! Please check your collector configuration!"
Oct 18 16:06:52 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:52+02:00" level=error msg="[filebeat] Backend crashed, trying to restart 3/3"
Oct 18 16:06:57 gm-01-c /usr/bin/graylog-collector-sidecar[16380]: time="2016-10-18T16:06:57+02:00" level=info msg="[filebeat] Starting (exec driver)"
Oct 18 16:06:57 gm-01-c graylog-collector-sidecar: time="2016-10-18T16:06:57+02:00" level=info msg="[filebeat] Starting (exec driver)"
The Server Logfile did not give any Error. User should get the message that the creation of this Sidecar and the register was not successfull.
I started with a new Graylog installation, empty database and thus without any collector configuration.
Apart from #36, I am also seeing that the filebeat and nxlog configurations are not updated once I created a configuration in the server. Only after restarting the sidecar process the new configurations get fetched.
I guess there is an issue with the new configuration caching system. To reproduce it, it should be enough to start a sidecar with a tag for which no configuration exists.
This is a multi node setup of Graylog 2.2.0-alpha.5-SNAPSHOT+1720c48 with sidecar 5e577167b55db1bdd4d99c05cacfe0d2fe13593a.
The plugin appears to be available source only.
The instructions imply availability as a jar (pre-built) for deploy.
Is the plugin available as a pre-built jar?
Thanks,
Rob
When using PT60S
as update interval, the sidecar gets the correct value of 60
seconds. When using PT1M
, the sidecar gets a wrong value of 0
seconds.
The sidecar details page should be adapted and reviewed.
Add a view of the generated config file under the collector status page so it can be easily viewed and verified centrally. Maybe a 'view config' button next to the current restart button?
Add process actions i.e. start, stop, restart.
From @silenceper on September 14, 2017 8:22
I use filebeat to collect message, but just specified the file path is too simple for me.
so i want to use the "exclude_files" properties.
Copied from original issue: Graylog2/graylog2-server#4145
As an admin of Graylog environments where installation and configuration of the Sidecar is delegated to various teams, I would like the ability to view a collector's IP address alongside the other collector status information. This would be useful in cases where a collector may not have an accurate name (e.g. "localhost") and/or I need to assist with troubleshooting an issue.
I have set up a logger for org.graylog2.rest.accesslog
, so I can dig in to get an IP if absolutely needed, but it would be much more convenient to have that info displayed with the other collector info.
The new CollectorSystemConfiguration
values need to be @Nullable
to ensure backwards compatibility.
2016-10-17 13:30:55,511 ERROR: org.graylog2.cluster.ClusterConfigServiceImpl - Error while deserializing payload
java.lang.IllegalArgumentException: Can not construct instance of org.graylog.plugins.collector.system.CollectorSystemConfiguration, problem: Missing required properties: collectorUpdateInterval
at [Source: N/A; line: -1, column: -1]
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3580) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3499) ~[jackson-databind-2.8.3.jar:2.8.3]
at org.graylog2.cluster.ClusterConfigServiceImpl.extractPayload(ClusterConfigServiceImpl.java:94) [classes/:?]
at org.graylog2.cluster.ClusterConfigServiceImpl.get(ClusterConfigServiceImpl.java:110) [classes/:?]
at org.graylog2.cluster.ClusterConfigServiceImpl.get(ClusterConfigServiceImpl.java:120) [classes/:?]
at org.graylog2.cluster.ClusterConfigServiceImpl.getOrDefault(ClusterConfigServiceImpl.java:125) [classes/:?]
at org.graylog.plugins.collector.system.CollectorSystemConfigurationSupplier.get(CollectorSystemConfigurationSupplier.java:45) [classes/:?]
at org.graylog.plugins.collector.system.CollectorSystemConfigurationSupplier.get(CollectorSystemConfigurationSupplier.java:25) [classes/:?]
at org.graylog.plugins.collector.periodical.PurgeExpiredCollectorsThread.doRun(PurgeExpiredCollectorsThread.java:83) [classes/:?]
at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:77) [classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_101]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_101]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of org.graylog.plugins.collector.system.CollectorSystemConfiguration, problem: Missing required properties: collectorUpdateInterval
at [Source: N/A; line: -1, column: -1]
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:268) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1441) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:468) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:487) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:276) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:224) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:471) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1196) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3575) ~[jackson-databind-2.8.3.jar:2.8.3]
... 16 more
Caused by: java.lang.IllegalStateException: Missing required properties: collectorUpdateInterval
at org.graylog.plugins.collector.system.AutoValue_CollectorSystemConfiguration$Builder.build(AutoValue_CollectorSystemConfiguration.java:144) ~[classes/:?]
at org.graylog.plugins.collector.system.CollectorSystemConfiguration.create(CollectorSystemConfiguration.java:56) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]
at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:117) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:274) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:224) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:471) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1196) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148) ~[jackson-databind-2.8.3.jar:2.8.3]
at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3575) ~[jackson-databind-2.8.3.jar:2.8.3]
... 16 more
2016-10-17 13:30:55,517 ERROR: org.graylog2.cluster.ClusterConfigServiceImpl - Couldn't extract payload from cluster config (type: org.graylog.plugins.collector.system.CollectorSystemConfiguration)
Is it possible to include support for symlinks at graylog-plugin-collector/src/web/collector-configuration/EditInputFields.jsx?
Or is it any workaround to get symlinks enabled?
For reference, this is the issue I'm facing https://discuss.elastic.co/t/how-to-enable-symlinks/126852
Thx!
The nxlog GELF output by default cuts off messages (short_message
) at 64 characters which might be unintuitive to most users.
The ShortMessageLength
setting for nxlog should explicitly be set to a higher (or configurable value) to avoid incomplete messages.
From https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_gelf
ShortMessageLength
This optional directive can be used to specify the length of the short_message field. This defaults to 64 if the directive is not explicitly specified. If the field short_message or ShortMessage is present, it will not be truncated.
https://nxlog.co/question/1037/short-messages-truncated-64-chars
The Type
column of the Beats Outputs table shows something indicating [FileBeat] Beats output
or [WinLogBeat] Beats output
, or the Create/Edit Output form's Type
select values indicate logstash
.
The Type
column of the Beats Outputs table shows logstash
while the Type
select on the Create/Edit Output form shows either [FileBeat] Beats output
or [WinLogBeat] Beats output
.
Create Output
Type
of either [FileBeat] Beats output
or [WinLogBeat] Beats output
Type
column shows logstash
logstash
wasn't mentioned on the Create/Edit form, so it was a bit odd to see it in the Beats Output table. I assume this is just multiple meanings of Type
(protocol + output type).
Where could I find the .jar file?
Creating a Beats snippet which extends on multiple lines does not work on Windows.
For example, the following snippets:
winlogbeat:
event_logs:
- name: Application
ignoreolder: 168h
- name: System
- name: Security
produces (on Windows) the following one-liner:
winlogbeat: event_logs: - name: Application ignoreolder: 168h - name: System - name: Security
which clearly breaks the Windows client.
Checking the source code, I think the problematic entry is here: it should use the platform-specific newline escape rather than hardcoding \n
When starting with a fresh MongoDB database, the collector configuration resource throws a NullPointerException.
2016-11-09 10:36:26,897 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
java.lang.NullPointerException: null
at org.graylog.plugins.collector.configurations.rest.resources.CollectorConfigurationResource.getConfiguration(CollectorConfigurationResource.java:110)
~[?:?]
at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_111]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_111]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog
.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?
]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:
?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.j
ava:160) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar
:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
Dependabot couldn't authenticate with https://oss.sonatype.org/content/repositories/releases.
You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.
You can mention @dependabot in the comments below to contact the Dependabot team.
Check if anything is using the fully qualified class names before changing the packages. We might need migrations to fix this.
Examples:
Also move all open issues to the new repo.
Sidecars are removed/marked as inactive after some time if there was no status update. It would be nice to first mark it with the status unknown
(no update from the node for a short time) and then after a second period delete/inactivate the node.
I've found two issues in Beta 1 of 2.1 (I have not put beta 2 in so apologies if this was already addressed but didn't see it). One I think is a bug, one is... an inconvenient feature perhaps.
The first I'd call a bug is that the text entered in the "arbitrary text" is inserted without a new line following it. Consider the "input" section. If you put a line of code, without adding an empty line, the resulting code in the config file is:
Exec If $SourceName == 'Microsoft-Windows-StorDiag' drop();</Input>
The trailing close of the input tag is the a syntax error. I can't think of any circumstance where you want code with the closing tag appended before line end, so you will cut down on a lot of (fairly hidden) errors if you ensure the arbitrary text ends in a new line.
The other issue is that the section is hidden (initially collapsed). I get that you did that to reduce casual use and mistakes, but once it has content, it should always be shown expanded. Otherwise if you go into the dialog, it is very easy to get confused and think there is no text there. I would suggest that if you want it initially collapsed, only do so when it is empty.
Thanks,
Linwood
The configuration modal of a collector became quite large over time. An extra configuration page for each collector would improve usability. Additionally more space would allow to provide more help on each configuration option.
Allow users to export configuration and log collectors through content packs.
It would be great to be able to download the sidecar binary directly from a graylog-server
process through the REST API and have it ship a configuration file that already points to the right graylog-server
for the first check in.
As we designed in mockups.
The graylog web interface is displaying messages from the filebeat input almost 15 minutes after being processed. I don't have this problem with the syslog udp input. I have checked the sever's, user's and web browser time configurations and they are all the same. The elasticsearch server is separate but I'm using an NTP server to sync the time.
collector_sidecar.yml:
server_url: http://graylogserver_ip:9000/api/
update_interval: 30
tls_skip_verify: true
send_status: true
list_log_files:
- /var/log
node_id: graylog-collector-sidecar-nodename
collector_id: file:/etc/graylog/collector-sidecar/collector-id
cache_path: /var/cache/graylog/collector-sidecar
log_path: /var/log/graylog/collector-sidecar
log_rotation_time: 86400
log_max_age: 604800
tags:
- syslog
- linux
backends:
- name: nxlog
enabled: false
binary_path: /usr/bin/nxlog
configuration_path: /etc/graylog/collector-sidecar/generated/nxlog.conf
- name: filebeat
enabled: true
binary_path: /usr/bin/filebeat
configuration_path: /etc/graylog/collector-sidecar/generated/filebeat.yml
filebeat version 5.1.1 (amd64), libbeat 5.1.1
Graylog Collector Sidecar version 0.1.0 (e14968e) [go1.7.4/amd64]
Graylog-2.2
The form state is not reset when switching between Beats and NXLog on the Collector Configuration page (/system/collectors/configurations/ID
). This is definitely minor... I only noticed it when I realized I was in the Beats section rather than NXLog.
/system/collectors/configurations/
and create a new collector configurationThe NXLog GELF TLS output should work without any cert files given.
I'm running Graylog v2.2.2 inside a docker container. I have two plugins running, beats input and collector. Both plugins show up on the configuration page.
I have a sidecar running on the physical host. However the option to configure collector sidecar presumably under http://ip:9000/system/collectors does not exist. Is this a configuration mistake somewhere? Do I need to enable this plugin with additional options?
If a collector configuration has outputs/inputs/snippets for one collector type (e.g. NXLog) but not any other types (e.g. Beats) the sub-tab for that populated type should be pre-selected when the page loads.
The "Beats" sub-tab is always selected first.
This is minor, but a bit annoying as we (currently) don't use Beats.
Setting a collector ID is no longer under our control, so it depends on users' configuration. We need to discuss if we want to keep this functionality (and how) or delete it.
The class RestBoolean
seems no longer necessary, as the corresponding Javascript store always passes true
now.
It just needlessly complicates the intended resource, making it harder to follow, boolean flags should simply use boolean datatypes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.