Giter Club home page Giter Club logo

graphite's Introduction

Description

Installs and configures Graphite http://graphite.wikidot.com/

Consult the Graphite documentation for more information:

Requirements

  • Ubuntu 10.04 / Ubuntu 12.04
  • Debian
  • RHEL and derivatives (Centos, Amazon Linux, Oracle Linux, Scientific Linux)
  • Fedora

Attributes

  • node['graphite']['version'] - version of graphite to install (defaults to 0.9.10)
  • node['graphite']['password'] - password for graphite root user (default to change_me and is only used if encrypted databag isn't)
  • node['graphite']['chef_role'] - chef role name for graphite instances, used by the federated recipe (defaults to "graphite")
  • node['graphite']['url'] - url of the graphite server (defaults to graphite)
  • node['graphite']['url_aliases'] - array of url aliases (defaults to nil)
  • node['graphite']['listen_port'] - port to listen on (defaults to 80)
  • node['graphite']['base_dir'] = "/opt/graphite"
  • node['graphite']['doc_root'] = "/opt/graphite/webapp"
  • node['graphite']['storage_dir'] = "/opt/graphite/storage"
  • node['graphite']['django_root'] = "@DJANGO_ROOT@" - configurable path to your django installation
  • node['graphite']['timezone'] - set the timezone for the graphite web interface, defaults to America/Los_Angeles
  • node['graphite']['whisper']['uri'] - download url for whisper
  • node['graphite']['whisper']['checksum'] - checksum of the whisper download
  • node['graphite']['encrypted_data_bag']['name'] - the name of the encrypted data bag containing the default password for the graphite "root" user. If this attribute is set it will not use node['graphite']['password'].

carbon-cache.py attributes

  • node['graphite']['storage_schemas'] - an array with retention rates for storing metrics, used to generate the storage-schemas.conf file (see the example below)
  • node['graphite']['storage_aggregation'] - an array with rules to configure how to aggregate data to lower-precision retentions, used to generate the storage-aggregation.conf file
  • node['graphite']['carbon']['uri'] - download url for carbon
  • node['graphite']['carbon']['checksum'] - checksum for the carbon download
  • node['graphite']['carbon']['line_receiver_interface'] - line interface IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['line_receiver_port'] - line interface port (defaults to 2003)
  • node['graphite']['carbon']['enable_udp_listener'] - set this to "True" to enable the UDP listener (defaults to "False")
  • node['graphite']['carbon']['udp_receiver_interface'] - line interface IP for UDP listener (defaults to 0.0.0.0)
  • node['graphite']['carbon']['udp_receiver_port'] - line interface port for UDP listener (defaults to 2003)
  • node['graphite']['carbon']['pickle_receiver_interface'] - pickle receiver IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['pickle_receiver_port'] - pickle receiver port (defaults to 2004)
  • node['graphite']['carbon']['use_insecure_unpickler'] - set this to "True" to use the old-fashioned insecure unpickler (defaults to "False")
  • node['graphite']['carbon']['cache_query_interface'] - cache query IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['cache_query_port'] - cache query port (defaults to 7002)
  • node['graphite']['carbon']['use_flow_control'] - set this to "False" to drop datapoints received after the cache reaches MAX_CACHE_SIZE (defaults to "True")
  • node['graphite']['carbon']['max_cache_size'] - max size of the carbon cache (defaults to "inf")
  • node['graphite']['carbon']['max_creates_per_second'] - max number of new metrics to create per second (defaults to "inf")
  • node['graphite']['carbon']['max_updates_per_second'] - max updates to carbon per second (defaults to "1000")
  • node['graphite']['carbon']['log_whisper_updates'] - log updates to whisper (defaults to "False")
  • node['graphite']['carbon']['whisper_autoflush'] - set this option to "True" if you want whisper to write synchronously (defaults to "False")
  • node['graphite']['carbon']['service_type'] - init service to use for carbon (defaults to runit)
  • node['graphite']['carbon'][instances] - array of hashes to configure more instances additionally to the default ones. You can put as many as you want to this array. (see the example below)

carbon-relay.py attributes

  • node['graphite']['relay_rules'] - an array with relay rules for sending metrics to a certain backends, used to generate the relay-rules.conf file (see the example below)
  • node['graphite']['carbon']['relay']['line_receiver_interface'] - line interface IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['relay']['line_receiver_port'] - line interface port (defaults to 2013)
  • node['graphite']['carbon']['relay']['pickle_receiver_interface'] - pickle receiver IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['relay']['pickle_receiver_port'] - pickle receiver port (defaults to 2014)
  • node['graphite']['carbon']['relay']['relay_method'] - choose between consistent-hashing and rules (defaults to "rules")
  • node['graphite']['carbon']['relay']['replication_factor'] - used to replicate datapoint data to more than one machine (defaults to 1)
  • node['graphite']['carbon']['relay']['destinations'] - list of carbon daemons to send metrics to
  • node['graphite']['carbon']['relay']['max_datapoints_per_message'] - maximum datapoints to send in a message between carbon daemons (defaults to 500)
  • node['graphite']['carbon']['relay']['max_queue_size'] - maximum queue of messages used to comunicate to other carbon daemons (defaults to 10000)
  • node['graphite']['carbon']['relay']['use_flow_control'] - set this to "False" to drop datapoints received after the cache reaches MAX_CACHE_SIZE (defaults to "True")
  • node['graphite']['carbon']['relay'][instances] - array of hashes to configure more instances additionally to the default ones. You can put as many as you want to this array. (see the example below)

carbon-aggregator.py attributes

  • node['graphite']['aggregation_rules'] - an array with rules that allow you to add several metrics together, used to generate the aggregation-rules.conf file (see the example below)
  • node['graphite']['carbon']['aggregator']['line_receiver_interface'] - line interface IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['aggregator']['line_receiver_port'] - line interface port (defaults to 2023)
  • node['graphite']['carbon']['aggregator']['pickle_receiver_interface'] - pickle receiver IP (defaults to 0.0.0.0)
  • node['graphite']['carbon']['aggregator']['pickle_receiver_port'] - pickle receiver port (defaults to 2024)
  • node['graphite']['carbon']['aggregator']['destinations'] - list of carbon daemons to send metrics to
  • node['graphite']['carbon']['aggregator']['replication_factor'] - used to add redundancy to your data by replicating every datapoing to more than one machinne (defaults to 1)
  • node['graphite']['carbon']['aggregator']['max_queue_size'] - maximum queue of messages used to comunicate to other carbon daemons (defaults to 10000)
  • node['graphite']['carbon']['aggregator']['use_flow_control'] - set this to "False" to drop datapoints received after the cache reaches MAX_CACHE_SIZE (defaults to "True")
  • node['graphite']['carbon']['aggregator']['max_datapoints_per_message'] - maximum datapoints to send in a message between carbon daemons (defaults to 500)
  • node['graphite']['carbon']['aggregator']['max_aggregation_intervals'] - sets how many datapoints the aggregator remembers for each metric (defaults to 5)

graphite-web attributes

  • node['graphite']['web']['uri'] - download url for the graphite web ui
  • node['graphite']['web']['checksum'] - checksum for the graphite web ui download
  • node['graphite']['web']['debug'] - debug mode (defaults to "False")
  • node['graphite']['web']['admin_email'] - admin contact email (defaults to "[email protected]")
  • node['graphite']['web']['cluster_servers'] - IP address (and optionally port) of the webapp on each remote server in the cluster
  • node['graphite']['web']['carbonlink_hosts'] - list the IP address, cache query port and instance name of each carbon cache instance on the local machine
  • node['graphite']['web_server'] - defaults to apache. Anything else will use uwsgi instead of apache
  • node['graphite']['user_account'] - user (default node['apache']['user'])
  • node['graphite']['group_account'] - group (default node['apache']['group'])
  • node['graphite']['create_user']- should the user be created, boolean (defaults to false)
  • node['graphite']['ssl']['enabled'] - enable ssl in the apache2 vhost
  • node['graphite']['ssl']['cipher_suite'] - the cipher suite to use if ssl is enabled
  • node['graphite']['ssl']['certificate_file'] - the path to the certificate file if ssl is enabled
  • node['graphite']['ssl']['certificate_key_file'] - the path to the vertificate key file if ssl is enabled
  • node['graphite']['ssl']['ca_certificate_file'] - the path to the intermediate CA certificate file if ssl is enabled. Leave blank if you don't have or need one.
  • node['graphite']['apache']['basic_auth']['enabled'] - enable basic auth in the apache2 vhost to require authentication for access to web interface (defaults to false)
  • node['graphite']['apache']['basic_auth']['file_path'] - location of htpasswd file for basic auth (defaults to node['graphite']['doc_root']/htpasswd)
  • node['graphite']['apache']['basic_auth']['user'] - username for basic auth
  • node['graphite']['apache']['basic_auth']['pass'] - password for basic auth
  • node['graphite']['uwsgi_socket'] - the socket to bind uwsgi process to (only needed if using uwsgi)

database settings

  • default['graphite']['web']['database']['NAME'] - the database name defaults to sqlite database in the storage folder
  • default['graphite']['web']['database']['ENGINE'] - the django database engine that will be used. The default is sqlite. If you want to use mysql set this to django.db.backends.mysql
  • default['graphite']['web']['database']['USER'] - database username leave this blank if you're using sqlite
  • default['graphite']['web']['database']['PASSWORD'] - database password leave this blank if you're using sqlite.
  • default['graphite']['web']['database']['HOST'] - database host leave this blank if you're using sqlite.
  • default['graphite']['web']['database']['PORT'] - database port leave this blank if you're using sqlite.

ldap settings

  • default['graphite']['web']['ldap']['SERVER'] - ldap server you want to use
  • default['graphite']['web']['ldap']['BASE_USER'] - the base dn of the user graphite web should use
  • default['graphite']['web']['ldap']['BASE_PASS'] - password for the base dn user
  • default['graphite']['web']['ldap']['USER_QUERY'] - the ldap query to find the user by name. If you're using MS AD this should be "(sAMAccountName=%s)"
  • default['graphite']['web']['ldap']['SEARCH_BASE'] - the search base the query should run against

email settings

  • default['graphite']['web']['email']['BACKEND'] - django email backend "django.core.mail.backends.smtp.EmailBackend"
  • default['graphite']['web']['email']['HOST'] - the smtp host. This defaults to "localhost"
  • default['graphite']['web']['email']['PORT'] - the smtp port.
  • default['graphite']['web']['email']['HOST_USER'] - the smtp user. Just use this if you need to authenticate against smtp
  • default['graphite']['web']['email']['HOST_PASSWORD'] - the smtp password
  • default['graphite']['web']['email']['USE_TLS'] - if you want to use tls change this to true

storage_schemas example

node.default['graphite']['storage_schemas'] = [
  {
    'name' => 'carbon',
    'pattern' => /^carbon\./,
    'retentions' => '1m:10d'
  },
  {
    'name' => 'sensu',
    'pattern' => /^sensu\./,
    'retentions' => '1m:30d'
  },
  {
    'name' => 'everything_30s7d_15m1m',
    'match-all' => true,
    'retentions' => '30s:7d,15m:1m'
  }
]

relay_rules example

node.default['graphite']['relay_rules'] = [
  {
    'name' => 'example_pattern',
    'pattern' => /^mydata\.foo\..+/,
    'destinations' => [ '10.1.2.3', '10.1.2.4:2004', 'myserver.mydomain.com' ]
  },{
    'name' => 'example_default',
    'default' => true,
    'destinations' => [ '10.1.2.5:2004' ]
  }
]

aggregation_rules example

node.default['graphite']['aggregation_rules'] = [
  {
    'output_template' => '<env>.applications.<app>.all.requests',
    'frequency' => '60',
    'method' => 'sum',
    'input_pattern' => '<env>.applications.<app>.*.requests'
  },
  {
    'output_template' => '<env>.applications.<app>.all.latency',
    'frequency' => '60',
    'method' => 'sum',
    'input_pattern' => '<env>.applications.<app>.*.latency'
  },
]

storage_aggregation example

node.default['graphite']['storage_aggregation'] = [
  {
    'name' => 'all_min',
    'pattern' => '\.min$',
    'xFilesFactor' => '0.1',
    'aggregationMethod' => 'min'
  },
  {
    'name' => 'count',
    'pattern' => '\.count$',
    'xFilesFactor' => '0',
    'aggregationMethod' => 'sum'
  },
]

carbon cache instances example

node['graphite']['carbon']['instances'] = 
[
  {
    "instance_name" => "cache1-a",
    "options" => {
      "LINE_RECEIVER_PORT" => 2013,
      "UDP_RECEIVER_PORT" => 2013,
      "PICKLE_RECEIVER_PORT" => 2014,
      "CACHE_QUERY_PORT" => 7012
    }
  }
]

will create this config

[cache:cache1-a]
LINE_RECEIVER_PORT = 2013
UDP_RECEIVER_PORT = 2013
PICKLE_RECEIVER_PORT = 2014
CACHE_QUERY_PORT = 7012

carbon relay instances example

destinations = ["localhost:2014:cache1-a", "localhost:2024:cache1-b"]
node['graphite']['carbon']['relay']['instances'] = 
[
  {
    "instance_name" => "relay1-a",
    "options" => {
      "DESTINATIONS" => destinations.join(', '),
      "RELAY_METHOD" => "consistent-hashing",
      "REPLICATION_FACTOR" => 1,
      "LINE_RECEIVER_PORT" => 3003,
      "UDP_RECEIVER_PORT" => 3003,
      "PICKLE_RECEIVER_PORT" => 3004,
    }
  }
]

will create this config:

[relay:relay1-a]
LINE_RECEIVER_PORT = 3003
UDP_RECEIVER_PORT = 3003
PICKLE_RECEIVER_PORT = 3004
REPLICATION_FACTOR = 1
RELAY_METHOD = consistent-hashing
DESTINATIONS = localhost:2014:cache1-a, localhost:2024:cache1-b

Data Bags

This cookbook optionally uses an encrypted data bag to store the graphite password. If this data bag is not present the cookbook will use node['graphite']['password'] instead. To use the encrypted data bag set node['graphite']['encrypted_data_bag']['name'] with the name of the data bag you wish to use.

Helper Scripts

The following helper scripts are included in the graphite/bin directory:

  • whisper-clean-this-node.sh - this script cleans the whisper metrics that belong to other machines in the cluster. Usually used after synchronizing the storage/whisper directory. Uses the whisper-clean.py script internally

Usage

recipe[graphite] should build a stand-alone Graphite installation.

recipe[graphite::carbon_aggregator] builds the aggregation_rules and storage-aggregation config files, and configures the carbon-aggregator service.

graphite's People

Contributors

webframp avatar zuazo avatar tas50 avatar dje avatar gansbrest avatar lusis avatar michaelballantyne avatar chrisroberts avatar pdf avatar smith avatar mbabineau avatar igalarzab avatar rkrol avatar nviennot avatar luckymike avatar retr0h avatar jarosser06 avatar gudmundur avatar ruphin avatar rampire avatar drosenstark avatar kmcminn avatar jschneiderhan avatar obradovic avatar thetrevdev avatar svanzoest avatar milosgajdos avatar mattjalexander avatar kppullin avatar josephholsten avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.