shunwen / fluent-plugin-rename-key Goto Github PK

I installed the plugin. But when I try to use it, fluentd won't start up. Any ideas? I tried both gem install and td-agent-gem install

sudo td-agent-gem install fluent-plugin-rename-key
Successfully installed fluent-plugin-rename-key-0.2.0
Parsing documentation for fluent-plugin-rename-key-0.2.0
Done installing documentation for fluent-plugin-rename-key after 0 seconds
1 gem installed

Done installing documentation for msgpack, yajl-ruby, cool.io, sigdump, serverengine, http_parser.rb, thread_safe, tzinfo, tzinfo-data, strptime, fluentd, fluent-plugin-rename-key after 9 seconds
12 gems installed

When starting up td-agent, I get the following error

2016-07-14 17:47:42 +0000 [info]: adding filter pattern=".*" type="rename_key"
2016-07-14 17:47:42 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error="Unknown filter plugin 'rename_key'. Run 'gem search -rd fluent-plugin' to find plugins"

It is a problem if I use it as a filter but works if I use it as a match. I need to combine it with another match. Any ideas on how to do this?

<match *.**>
type rename_key
#remove_tag_prefix input.test
#append_tag renamed
rename_rule1 . _

The following doesn't work

<filter *.**>
type rename_key
#remove_tag_prefix input.test
#append_tag renamed
rename_rule1 . _

https://github.com/shunwen/fluent-plugin-rename-key#installation has no details beyond a dead link.

I can't find this plugin on https://www.fluentd.org/plugins

How do I install it?

EDIT: I found https://rubygems.org/gems/fluent-plugin-rename-key/versions/0.4.1 so I guess this

gem install fluent-plugin-rename-key

@shunwen

Could you push git tags v0.3.3, v0.3.4 and v0.4.0 to GitHub? I want to compare each versions.

• https://github.com/shunwen/fluent-plugin-rename-key/releases
• https://rubygems.org/gems/fluent-plugin-rename-key/versions

Thanks for your great gem!

fluent-plugin-rename-key requres ruby version >= 2.2 from f667339 .

However latest td-agent (v2.3.3) has ruby-v2.1.10.
https://docs.treasuredata.com/articles/td-agent-changelog#20160930-version-v233

$ /opt/td-agent/embedded/bin/ruby --version
ruby 2.1.10p492 (2016-04-01 revision 54464) [x86_64-linux]

Is it the intended change?

Best regards.

If I want to delete special characters, what is the easiest way to do it using this tool?

for example, I want to replicate the sed functionality of:

s/[^0-1a-zA-Z_]//g

that is, all characters that are not (alphanumeric and underscore) are removed //

I thought of a few ways, but the trouble is that the good stuff, the content we want to keep, would appear in an unknown number of placeholders ${md[1]} , ${md[2]}, ${md[3]}. ... I cannot see how to join these.

In Ruby, this would be like:
"myKey".gsub(/[^0-1a-zA-Z_]/, "")

Any help is appreciated!

Thanks!

Jeff

Hi,

just wondering what the current maintenance state of this project is.

It seems rather useful to circumvent Kibana's _-prefix restriction when dealing with systemd/docker/gelf/kubernetes logs.

I've been trying to use this plugin to rename all fields from uppercase to lowercase (systemd input plugin in Fluent Bit exports everything in uppercase) and it messes with some keys.