Comments (7)
And also want to reflect the parameters in the document:https://www.elastic.co/guide/en/logstash/2.3/plugins-outputs-webhdfs.html#plugins-outputs-webhdfs-message_format
from logstash-output-webhdfs.
Hello langke93 !
Thanks for your feedback and problem report. I will have a look at this problem in the next couple of days and get back to you as soon as I have some news.
Take care,
Björn
from logstash-output-webhdfs.
Hi again!
The configuration option 'message_format' was removed in commit 8f1cf04#diff-d807ea36853d4e0b3acf6ef50b87c259.
Please use the format option of the line output codec instead. E.g.:
output { webhdfs { host => ... .... codec => { line { format => "custom format: %{message}" } } } }
Still, we need to check why the documentation does not reflect the current code and sorry for the confusion, caused by this!
So thank you very much for pointing this out and for your commit! Still, as the default way should be to use the line codec with the format parameter, I will not merge your commit. Not because it's bad or anything ;)
Take care and all the best,
Björn
I will leave this issue open until we found out, why the documentations is not up to date.
from logstash-output-webhdfs.
output{
webhdfs{
host => "sea2"
standby_host => "sea3"
codec => {line { format => "custom format: %{message}"}}
}
}
tips:
The given configuration is invalid. Reason: Expected one of #, => at line 35, column 20 (byte 1427) after output {
webhdfs{
host => "sea2"
standby_host => "sea3"
codec => {line {:level=>:fatal}
And then I modify:
codec => {line =>{ format => "custom format: %{message}"}}
Then prompt :
Pipeline aborted due to error {:exception=>#<NoMethodError: undefined method `on_event' for ["line", {"format"=>"custom format: %{message}"}]:Array>
from logstash-output-webhdfs.
My bad ;)
The example I gave was invalid, as you correctly pointed out.
Correct example:
webhdfs{
host => "sea2"
standby_host => "sea3"
codec => line { format => "custom format: %{message}"}
}
Sorry for this mistake!
Take care,
Björn
from logstash-output-webhdfs.
Hello langke93 !
Could you please let me know if this new configuration does work for you?
If so, this issue could be closed.
Take care and have a nice weekend,
Björn
from logstash-output-webhdfs.
thanks,is ok
from logstash-output-webhdfs.
Related Issues (20)
- Compatibility with logstash 2.1? HOT 4
- Offline Installs - missing in plugin pack HOT 3
- Write errors when running vs HDP 2.3 (==Hadoop 2.7) HOT 21
- Idea/enhancement: support for HA namenodes HOT 7
- conf error HOT 2
- logstash-output-webhdfs不支持hdfs的ha模式 HOT 3
- WebHDFS:IOError HOT 6
- Failed to flush outgoing items {:outgoing_count=>1, :exception=>"WebHDFS::ServerError" HOT 4
- feature request: support for kerberos via keytab HOT 7
- [logstash.outputs.webhdfs ] Failed to flush outgoing items {:outgoing_count=>1, :exception=>"LogStash::Error" HOT 1
- Ticket Expired : Logstash + WebHdfs + Kerberos HOT 3
- 403 GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails) HOT 5
- write hdfs failed after the first time successfully written HOT 2
- Unnecessary failovers for HDFS namenode
- kerberos fail HOT 3
- why there need to check standby_host before create connect
- Add Azure Data Lake Storage (ADLS) compatibility
- Incomplete GZIP Block During Interrupted Writes Corrupts Files
- WebHDFS::ServerError: Failed to connect to host, end of file reached
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logstash-output-webhdfs.