ViaQ Fluentd docker container - can be used as either a local collector, or as an aggregator/formatter/normalizer, for feeding data into Elasticsearch.

Elasticsearch output

• ENABLE_ES - use Elasticsearch for output. Defaults to true
• LOGSTASH_PREFIX - Elasticsearch index name prefix. Defaults to viaq
• ES_HOST must be FQDN of ElasticSearch server. Defaults to viaq-elasticsearch.
• ES_PORT must be the port on which the ElasticSearch server is listening. Defaults to 9200.

Syslog input

• ENABLE_SYSLOG - use syslog (RFC5424) listener for tcp/udp input. Defaults to true.
• SYSLOG_LISTEN_PORT the port this rsyslog instance is listening for. both TCP and UDP. Defaults to 10514.
• SYSLOG_LISTEN_BIND_ADDR - ip address to bind to. Defaults to 0.0.0.0
• FLUENTD_SYSLOG_LOG_LEVEL - default is FLUENTD_LOG_LEVEL or warn

Forwarder input

• ENABLE_FORWARD - use fluentd forwarder listener (e.g. for fluent-cat) for tcp/udp input. Defaults to true.
• FLUENTD_FORWARD_INPUT_PORT - default is 24224
• FLUENTD_FORWARD_INPUT_BIND_ADDR - default is 0.0.0.0
• FLUENTD_FORWARD_INPUT_LOG_LEVEL - default is FLUENTD_LOG_LEVEL or warn

Journal input

• ENABLE_JOURNAL - read from systemd journal - default false
• JOURNAL_DIR - default is /run/log
• FLUENTD_JOURNAL_LOG_LEVEL - default is FLUENTD_LOG_LEVEL or warn

Tail/file input

• ENABLE_TAIL - default is false
• FLUENTD_TAIL_LOG_DIR - default /var/log - reads the file(s) messages* in that dir
• FLUENTD_TAIL_LOG_LEVEL - default is FLUENTD_LOG_LEVEL or warn

AMQP input

• ENABLE_AMQP_INPUT - read from AMQP queue - default true
• FLUENTD_AMQP_INPUT_URL - default is amqp://viaq-qpid-router:5672/viaq
• FLUENTD_AMQP_INPUT_LOG_LEVEL - default is FLUENTD_LOG_LEVEL or warn

Stdout output (for debugging)

• ENABLE_STDOUT - default false

Monitor http://docs.fluentd.org/articles/monitoring

• ENABLE_MONITOR - Defaults to true.
• MONITOR_PORT - default is 24220
• MONITOR_BIND_ADDR - default is 0.0.0.0

Debug http://docs.fluentd.org/articles/monitoring

• ENABLE_DEBUG - Defaults to false.
• DEBUG_PORT - default is 24230
• DEBUG_BIND_ADDR - default is 127.0.0.1

If you want to use fluentd with or as a normalizer, you must define the following:

• NORMALIZER_NAME - The string name of the normalize reported in the ES record as "pipeline_metadata":{"normalizer":{"name": "NORMALIZER_NAME"}}. This is a descriptive string used for searching and filtering.
• NORMALIZER_IP - not currently used
• NORMALIZER_HOSTNAME - hostname of the normalizer node/machine. This is reported in the ES record as "pipeline_metadata":{"normalizer":{"hostname": "NORMALIZER_HOSTNAME"}}.

Ruby version

• RUBY_SCL_VER - Is used to specify Ruby version from software collection. Currently supports rh-ruby22. By default Ruby from base Docker image is used (i.e. centos7 comes with Ruby 2.0.0p598).

In order to add own Fluentd configuration file please add the configuration files to a local directory and map in to /data docker volume. The following files are taken form the local directory: fluent.conf, config.d/*, patterns.d/* In case fluent.conf exists, the default config.d/*.conf is removed and not used in the container.

Using plain docker, default arguments:

# docker run -d -p 10514:10514/udp -p 24224:24224/udp -p 24220:24220 \
  -e FLUENTD_LOG_LEVEL=info --name viaq-fluentd viaq/fluentd

Use Ruby 2.2 from software collection:

# docker run -d -e RUBY_SCL_VER=rh-ruby22 ... --name viaq-fluentd viaq/fluentd

Using specified syslog listen host, fluentd config dir, normalizer configuration:

# docker run -d -p $syslog_listen_port:$syslog_listen_port/tcp \
  -p $syslog_listen_port:$syslog_listen_port/udp -v $local_dir:/data \
  -u $uid -e ES_HOST=$elasticsearchhost -e ES_PORT=$port \
  -e SYSLOG_LISTEN_PORT=$syslog_listen_port \
  -e NORMALIZER_NAME=container-rsyslog8.17 -e NORMALIZER_IP=$normalizer_ip \
  -e LOGSTASH_PREFIX=v2016.03.10.0-viaq --name viaq-fluentd viaq/docker-fluentd