Logstash - datadog - Logs missing about logstash-output-datadog_logs HOT 9 CLOSED

The issue described here and in the linked support case is now solved. The broken pipe errors generally indicate a closed connection -the tcp log ingestion endpoint will shut down after x seconds of inactivity. For this situation, HTTP forwarding would be better suited.

If you are experiencing missing logs due to broken pip errors using the plugin, currently you can use the default Logstash HTTP plugin which might look something like:

output {
  http {
    url => "https://http-intake.logs.datadoghq.com/v1/input/<DATADOG_API_KEY>?host=<HOST>&service=<SERVICE>&ddsource=<SOURCE>"
    http_method => "post"
    headers => ["Content-Type", "application/json"]
}
}

If you'd like to be added to the feature request for an HTTP forwarder in the Datadog output plugin, contact Datadog Support.

from logstash-output-datadog_logs.

Thanks @LawrenceLin690 your workaround worked for me.

from logstash-output-datadog_logs.

@gaetan-deputier Upgraded my plugin from 0.3.1 to 0.4.1 and it appears to have solved this issue for me. SS shows the past hour of a process that runs every 5 mins, upgraded plugin halfway thru and all logs appear to be coming in correctly now.
Probably safe to close!

from logstash-output-datadog_logs.

Hello @bl85774 ,

Thanks for reporting this issue and providing the full error message.
At a glance, it seems that there is indeed an issue with the multi_receive configuration and somehow it generate an error when sending the logs.

We will definitely look into it and come back to you.

In the mean time, do you already have a support ticket open on Datadog? (just to link it with this issue if that was the case)
And would you have some more information about your Logstash configuration so we can replicate with the exact same setup?

Thanks a lot

from logstash-output-datadog_logs.

from logstash-output-datadog_logs.

Thanks a lot,

I have linked both this issue and the support case. I believe we will start discussing more in details about this in the support ticket.

Just for me to double check, the apikey you shared as been edited to not reflect exactly the value right? As we strongly recommend to never share publicly an API KEY.

from logstash-output-datadog_logs.

Hi,

Thank you for your answer....

I hope we will discussing more in the Datadog support ticket.

The apikey is not reflect the right value....

from logstash-output-datadog_logs.

I observed a similar error, I was wondering what versions of logstash are supported? Is it all versions? I am observing broken pipe errors on logstash 6.8 and it has made the dd plugin not usable.

from logstash-output-datadog_logs.

Safe to close?

from logstash-output-datadog_logs.