add support for format_json about fluent-plugin-webhdfs HOT 10 CLOSED

You can get output you want by output_include_tag false and output_include_time false.
I'll add this option on README.

from fluent-plugin-webhdfs.

Update on 0bbbb5f

from fluent-plugin-webhdfs.

Thanks.
One more thing, is it possible to insert time and tag inside the record?

from fluent-plugin-webhdfs.

I'm sorry that it's not supported by current versions of fluent-plugin-webhdfs.
I think that it seems useful for some users, so I'm waiting patches!

from fluent-plugin-webhdfs.

I'll try to make one.

One thing, where do i find output_include_tag and output_include_time params and config?

from fluent-plugin-webhdfs.

That features are from fluent-mixin-plaintextformatter.
See: https://github.com/tagomoris/fluent-mixin-plaintextformatter

FYI: mixins to insert time and tag are SetTimeKeyMixin and SetTagKeyMixin in fluentd core:
https://github.com/fluent/fluentd/blob/master/lib/fluent/mixin.rb#L129
For example: https://github.com/fluent/fluent-plugin-s3/blob/master/lib/fluent/plugin/out_s3.rb#L22

from fluent-plugin-webhdfs.

I support _tag and _time in the json message! hope i'll find free time to submit a patch

from fluent-plugin-webhdfs.

@asyd 👍

from fluent-plugin-webhdfs.

I succeeded to get tag and time in the json message using reformer plugin:

<source>
  type syslog
  port 5140
  bind 0.0.0.0
  tag system
</source>

<match system.**>
  type record_reformer
  output_tag syslog.${host}
  <record>
    time ${time}
    message ${message}
    facility ${tag_suffix[1]}
  </record>
</match>

<match syslog.**>
  type forest
  subtype webhdfs
  remove_prefix syslog
  <template>
    namenode 127.0.0.1:50070
    path /fluentd/${tag}/%Y%m%d.log
    flush_interval 10s
    username nobody
    output_include_time false
    output_include_tag  false
  </template>
</match>

from fluent-plugin-webhdfs.

Open another issue for more ideas and patches.

from fluent-plugin-webhdfs.