Giter Club home page Giter Club logo

telegraf-cookbook's Introduction

telegraf

Build Status

Cookbook to install and configure telegraf

This was influenced by SimpleFinanace/chef-influxdb

Note: Some inputs will require other packages be installed and that is out of scope for this cookbook. ie. [netstat] requires lsof

Tested Platforms

  • CentOS 6.8 and 7.3
  • Ubuntu 15.04 and 16.04
  • Amazon Linux
  • Windows 2012R2
  • Mac OSX 10.12.x and 10.13.x

Requirements

  • Chef 12.5+

Usage

This cookbook can be used by including telegraf::default in your run list and settings attributes
as needed. Alternatively, you can use the custom resources directly.

Attributes

Key Type Description Default
node['telegraf']['version'] String Version of telegraf to install, nil = latest '0.10.0-1'
node['telegraf']['config_file_path'] String Location of the telgraf main config file '/etc/telegraf/telegraf.conf'
node['telegraf']['config'] Hash Config variables to be written to the telegraf config {'tags' => {},'agent' => {'interval' => '10s','round_interval' => true,'flush_interval' => '10s','flush_jitter' => '5s'}
node['telegraf']['outputs'] Hash telegraf outputs {'influxdb' => {'urls' => ['http://localhost:8086'],'database' => 'telegraf','precision' => 's'}}
node['telegraf']['include_repository'] [TrueClass, FalseClass] Whether or not to pull in the InfluxDB repository to install from. true
node['telegraf']['inputs'] Hash telegraf inputs {'cpu' => {'percpu' => true,'totalcpu' => true,'drop' => ['cpu_time'],},'disk' => {},'io' => {},'mem' => {},'net' => {},'swap' => {},'system' => {}}
node['telegraf']['perf_counters'] Hash telegraf performance counters {{ 'Processor' => { 'Instances' => ['*'] 'Counters' => ['% Idle Time','% Interrupt Time','% Privileged Time','% User Time','% Processor Time','% DPC Time',],'Measurement' => 'win_cpu','IncludeTotal' => true}}

Custom Resources

telegraf_install

Installs telegraf and configures the service. Optionally specifies a version, otherwise the latest available is installed. The default installation type is package. For windows this defaults to chocolatey_package. If Chocolatey is not installed the resource will install this for you. Alternatively you can use the installation type file which will download and install the package.

telegraf_install 'default' do
  install_version '0.10.0-1'
  install_type 'package'
  action :create
end

telegraf_config

Writes out the telegraf configuration file. Optionally includes outputs and inputs.

telegraf_config 'default' do
  path node['telegraf']['config_file_path']
  config node['telegraf']['config']
  outputs node['telegraf']['outputs']
  inputs node['telegraf']['inputs']
  perf_counters node['telegraf']['perf_counters']  
end

telegraf_outputs

Writes out telegraf outputs configuration file. You can call this several times to create multiple outputs config files.

telegraf_outputs 'default' do
  outputs node['telegraf']['outputs']
end

telegraf_inputs

Writes out telegraf inputs configuration file.

telegraf_inputs 'default' do
  inputs node['telegraf']['inputs']
end

You can call this several times to create multiple inputs config files. You'll need to specify different names for each telegraf_inputs resource, so they'll create separate config files.

For example, to add the nginx input:

node.default['telegraf']['nginx'] = {
  'nginx' => {
    'urls' => ['http://localhost/status']
  }
}

telegraf_inputs 'nginx' do
  inputs node['telegraf']['nginx']
  service_name 'default'
  reload true
  rootonly false
end

Note that there are three optional parameters for this resource that could've been left out in this case:

  • service_name [default: 'default'] if you need to override which service should be restarted when the config changes;
  • reload [default: true] whether to restart the service when the config changes;
  • rootonly [default: false] whether to restrict access to the config file so it's not world readable;

telegraf_perf_counters

Writes out telegraf performance counters configuration file for windows hosts only.

telegraf_perf_counters 'default' do
  perf_counters node['telegraf']['perf_counters']
end

You can call this several times to create multiple performance counter config files. You'll need to specify different names for each telegraf_perf_counters resource, so they'll create separate config files.

For example, to add a network interface performance counter:

node.default['telegraf']['network_interface'] = {
  'Network Interface' => {
    'Instances' => ['*'],
    'Counters' => [
      "Bytes Received/sec",
      "Bytes Sent/sec",
      "Packets Received/sec",
      "Packets Sent/sec",
    ],
    'Measurement' => 'win_net',
  },
}

telegraf_perf_counters 'network_interface' do
  perf_counters node['telegraf']['network_interface']
  service_name 'default'
  reload true
end

For more examples visit influxdata/telegraf/plugins/inputs/win_perf_counters

Note that there are three optional parameters for this resource that could've been left out in this case:

  • service_name [default: 'default'] if you need to override which service should be restarted when the config changes;
  • reload [default: true] whether to restart the service when the config changes;

License and Authors

Copyright (C) 2015-2017 NorthPage

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

telegraf-cookbook's People

Contributors

bli111 avatar brennentsmith avatar codylewandowski avatar don-code avatar dshkuratov avatar fishnix avatar frank-m avatar greglu avatar kjschnei001 avatar phromo avatar plork avatar sauraus avatar scottmlikens avatar snkutlu avatar tenyo avatar thesas avatar zvymazal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

telegraf-cookbook's Issues

Latest release is causing invalid <name>_output.conf files to be created

Hi guys,

As the title says the latest cookbook version is creating invalid config files. If you look at the below:

[[outputs]] [outputs.influxdb] database = "telegraf" ......

The [[outputs]] line is invalid and is not present in the accompanying _inputs.conf:

[inputs.cpu] drop = ["cpu_time"] percpu = true totalcpu = true [inputs.mem] [inputs.net] [inputs.netstat] [inputs.processes] [inputs.procstat] .......

I'm not convinced it is the cookbook code as this has not changed in the last few days. Furthermore the usage is standard as per you README. We only pinned the minimum & not maximum version of your cookbook.

Whilst I haven't delved too much into the why it would seem it is something to do with the TOML gem you are now using.

Thanks :)

chef v17 Deprecation warnings

chef v17 is reporting following deprecation warnings with telegraf-cookbook v0.12.0.
Please let me know if you want me to contribute to fixing this issue.

OS: CentOS 7.9
Chef Infra Client: 17.3.48

  The  resource in the telegraf cookbook should declare `unified_mode true` at 5 locations:
    - /var/chef/cache/cookbooks/telegraf/resources/config.rb
    - /var/chef/cache/cookbooks/telegraf/resources/inputs.rb
    - /var/chef/cache/cookbooks/telegraf/resources/install.rb
    - /var/chef/cache/cookbooks/telegraf/resources/outputs.rb
    - /var/chef/cache/cookbooks/telegraf/resources/perf_counters.rb
   See https://docs.chef.io/deprecations_unified_mode/ for further details.

Service is not restarted delayed

I'm trying to use the cookbook to install telegraf 0.10, and i use the default recipe, plus some telegraf_inputs and telegraf_outputs resource, and even if in the code the notifies are marked as :delayed, it restarts the service immediatly.

I'm not sure what's happening there, but it might be because each time the service is redefined in the resource.

It leads to failure on debian 8.2 where the service is restarted too often too quickly and systemd prevents it from restarting.

Any ideas on how to fix it?

Multiple Service processes

Hi all,

Having an issue whereby multiple processes are spawned from the service definitions created by the cookbook. It seems to be new behaviour although, confusingly your version hasn't changed since we started using Telegraf. Some more info:

service definition:

[Unit]
Description=The plugin-driven server agent for reporting metrics into InfluxDB
Documentation=https://github.com/influxdata/telegraf
After=network.target

[Service]
EnvironmentFile=-/etc/default/telegraf
User=micro
ExecStart=/usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d ${TELEGRAF_OPTS}
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
KillMode=process

[Install]
WantedBy=multi-user.target
Alias=telegraf.service

systemctl output:

telegraf.service - The plugin-driven server agent for reporting metrics into InfluxDB
   Loaded: loaded (/usr/lib/systemd/system/telegraf.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-06-30 14:41:02 UTC; 1 weeks 3 days ago
     Docs: https://github.com/influxdata/telegraf
 Main PID: 16983 (telegraf)
   CGroup: /system.slice/telegraf.service
           ├─15459 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
           └─16983 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d

ps output:

dkitchen  2268  0.0  0.0 112648   968 pts/0    S+   13:05   0:00 grep --color=auto telegraf
telegraf 15459  0.1  0.3 410960 27128 ?        Sl   Jun30  27:37 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
micro    16983  1.9  0.3 475952 23876 ?        Ssl  Jun30 300:00 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d

In some cases this is harmless but in some cases it seems to first spawn a 'zombie' process. When the second one comes up it cannot bind the port and we get no metrics. You guys have any ideas what could cause this?

Conflict with influxdb cookbook

Both cookbooks add https://repos.influxdata.com/ubuntu repository but with different options (influxdb use only stable branch, telegraf use stable and unstable). If use both cookbook it's rewriting apt source list every run.

Missing Inputs

Hi, we tested the latest Version. Our Attribute File for Inputs look like this

node.default['telegraf']['inputs'] = { 'cpu' => {'percpu' => true,'totalcpu' => true,'drop' => ['cpu_time']}, 'disk' => {}, 'io' => {}, 'mem' => {}, 'net' => {}, 'swap' => {} }

Only CPU Input is creted in the default_inputs.conf

[inputs.cpu]
drop = ["cpu_time"]
percpu = true
totalcpu = true

when my attribute file is defined like that
node.default['telegraf']['inputs'] = { 'cpu' => {'percpu' => true,'totalcpu' => true,'drop' => ['cpu_time']}, 'disk' => { 'mount_points' => ['/']} }

Then the default_inputs.conf looks like this

[inputs.cpu]
drop = ["cpu_time"]
percpu = true
totalcpu = true

[inputs.disk]
mount_points = ["/"]

So all { } entries will not be created . Please help.

Cheers Sven

Multiple socket_listener not possible

Sometimes you need multiple socket_listeners but this is not possible and may not be fixable with the toml converter in use. Just documenting the problem.

node['telegraf']['inputs']['socket_listener']['service_address'] = "udp://:2003"
node['telegraf']['inputs']['socket_listener']['service_address'] = "tcp://:2003"

edit: clarify problem

Chef-13 compatibility

* telegraf_install[default] action create
       ================================================================================
    Error executing action `create` on resource 'telegraf_install[default]'
    ================================================================================
    
    RuntimeError
    ------------
    I do not support your platform: amazon

The problem is this string - https://github.com/NorthPage/telegraf-cookbook/blob/master/resources/install.rb#L30

As for chef-13:

platform: amazon
platform_family: amazon

telegraf_install resource fails on Amazon Linux

On Amazon Linux the platform_family? is reported as rhel, but $releasever looks like 2015.09.1. The repo contains only CentOS releases 5, 6, and 7. I've been setting up the yum repo manually to use rhel/7 so the baseurl looks like:

https://repos.influxdata.com/rhel/7/\$basearch/stable

This works fine for me. Perhaps baseurl should be exposed as an attribute?

Missing yum resource repository in new yum version 5.0

Hello all,

I was failing my kitchen converge with this output:
Converging 3 resources
Recipe: build-essential::default
* build_essential[install_packages] action install
* yum_package[autoconf, bison, flex, gcc, gcc-c++, gettext, kernel-devel, make, m4, ncurses-devel, patch] action install
- install version 2.5.37-3.el7 of package flex

   Recipe: telegraf::default
     * telegraf_install[default] action create
       
       ================================================================================
       Error executing action `create` on resource 'telegraf_install[default]'
       ================================================================================
       
       NoMethodError
       -------------
       No resource or method named `yum_repository' for `Custom resource telegraf_install from cookbook telegraf action provider "default"'
       
       Cookbook Trace:
       ---------------
       /tmp/kitchen/cache/cookbooks/telegraf/resources/install.rb:31:in `block in class_from_file'
       
       Resource Declaration:
       ---------------------
       # In /tmp/kitchen/cache/cookbooks/telegraf/recipes/default.rb
       
        19: telegraf_install 'default' do
        20:   include_repository node['telegraf']['include_repository']
        21:   install_version node['telegraf']['version']
        22:   action :create
        23: end
        24: 
       
       Compiled Resource:
       ------------------
       # Declared in /tmp/kitchen/cache/cookbooks/telegraf/recipes/default.rb:19:in `from_file'
       
       telegraf_install("default") do
         action [:create]
         retries 0
         retry_delay 2
         default_guard_interpreter :default
         declared_type :telegraf_install
         cookbook_name "telegraf"
         recipe_name "default"
         include_repository true
         install_type "package"
       end

After digging, I found that the repository resource has been removed from yum 5.0 in their February update from yum 4.2. Has anyone else faced this issue? I was able to resolve by adding depends 'yum', '= 4.2' in my cookbook.

NP-2498 telegraf plugins and outputs shouldn't expect top level name

ie. This attribute/parameter to the plugins method in the telegraf_plugins resource:

default['telegraf']['plugins'] = {
  'plugins' => {
    'cpu' => {
      'percpu' => true,
      'totalcpu' => true,
      'drop' => ['cpu_time']
    },
    'disk' => {},
    'io' => {},
    'mem' => {},
    'net' => {},
    'swap' => {},
    'system' => {}
  }
}

should be:

default['telegraf']['plugins'] = {
  'cpu' => {
    'percpu' => true,
    'totalcpu' => true,
    'drop' => ['cpu_time']
  },
  'disk' => {},
  'io' => {},
  'mem' => {},
  'net' => {},
  'swap' => {},
  'system' => {}
}

The same is true for outputs.

NP-2492 Support telegraf 0.10.x

telegraf 0.10.x has brought some breaking changes. We need to update the cookbook to support those changes. Note: the database format also changed, but migrating databases should be out of scope.

TOML module breaking changes

PR #39 should fix this:

  * telegraf_config[default] action create
    * chef_gem[toml-rb] action install (up to date)

    ================================================================================
    Error executing action `create` on resource 'telegraf_config[default]'
    ================================================================================

    NoMethodError
    -------------
    undefined method `dump' for TOML:Module

    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/telegraf/resources/config.rb:46:in `block (2 levels) in class_from_file'
    /var/chef/cache/cookbooks/telegraf/resources/config.rb:45:in `block in class_from_file'

    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/telegraf_wrapper/recipes/_config.rb

     47: telegraf_config 'default' do
     48:   config node['telegraf']['config']
     49:   inputs node['telegraf']['inputs']
     50:   outputs node['telegraf']['outputs']
     51: end

    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/telegraf_wrapper/recipes/_config.rb:47:in `from_file'

    telegraf_config("default") do
      action [:create]
      default_guard_interpreter :default
      declared_type :telegraf_config
      cookbook_name "telegraf_wrapper"
      recipe_name "_config"
      config {"tags"=>{}, "agent"=>{"interval"=>"10s", "round_interval"=>true, "flush_interval"=>"10s", "flush_jitter"=>"5s"}}
      inputs {"cpu"=>{"percpu"=>false, "totalcpu"=>true, "collect_cpu_time"=>false}, "disk"=>{"ignore_fs"=>["tmpfs", "devtmpfs"]}, "diskio"=>{}, "kernel"=>{}, "mem"=>{}, "processes"=>{}, "swap"=>{}, "system"=>{}, "filestat"=>{"files"=>["/var/log/**.log"]}, "kernel_vmstat"=>{}, "net"=>{}, "netstat"=>{}}
      outputs {"influxdb"=>{"urls"=>["http://idb-01.admin.....:8086"], "retention_policy"=>"", "write_consistency"=>"any", "timeout"=>"5s", "username"=>"telegraf", "password"=>"*******", "database"=>"metrics", "user_agent"=>"telegraf"}}
    end

    System Info:
    ------------
    chef_version=13.1.31
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
    program_name=chef-client worker: ppid=16435;start=09:57:50;
    executable=/opt/chef/bin/chef-client

Version check is not working

The version check is not working as expected. The regex is looking for a v in front of the version, but this v is not part of the telegraf.exe --version output.

Tests failing due to outdated `chefdk` on Jenkins slave

Looks like tests are failing due to outdated chefdk on the jenkins slave:

  1) telegraf::default on Windows 2012r2 converges successfully
     Failure/Error: expect { chef_run }.to_not raise_error

       expected no Exception, got #<Fauxhai::Exception::InvalidPlatform: Could not find platform 'windows/2012r2' in any of the sources!> with backtrace:
         # ./spec/unit/recipes/default_spec.rb:33:in `new'
         # ./spec/unit/recipes/default_spec.rb:33:in `block (5 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:38:in `block (6 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:38:in `block (5 levels) in <top (required)>'
     # ./spec/unit/recipes/default_spec.rb:38:in `block (5 levels) in <top (required)>'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.