Giter Club home page Giter Club logo

aerospike-collectd's Introduction

aerospike-collectd

Aerospike plugin for collectd

Aerospike Monitoring Stack

For monitoring and alerting you should consider using the Prometheus and Grafana based Aerospike Monitoring Stack. This is the monitoring solution being developed by Aerospike.

Community Development

This repository has been turned over to the community. If you wish to contribute code, go ahead and clone this repo, modify the code, and create a pull request.

Active contributors can then ask to become maintainers for the repo. The wiki can similarly be modified by any code contributor who has been granted pull permissions.

Compatibility

Fully compatible with Aerospike Server 4.0 - 5.1.0.7 Although, only tested on 4.9.0.11, 5.0.0.11, and 5.1.0.7.

If you use a server version outside of 4.0 - 5.1.0.7 it should work fine but could be missing a few metrics.

Features

  • Service Level Stats (asinfo -v "statistics")
  • Namespace Stats (asinfo -v "namespace/NAMESPACE_NAME")
  • Set Stats (asinfo -v "sets/NAMESPACE_NAME/SET_NAME")
  • Bin Stats (asinfo -v "bins/NAMESPACE_NAME")
  • SIndex Stats (asinfo -v "sindex/NAMESPACE_NAME")
  • Latency Stats (asinfo -v "latency:") & (asinfo -v "latencies:") 5.1+
  • XDR Stats (asinfo -v "dc/DC_NAME") & (asinfo -v "get-stats:context=xdr;dc=DC_NAME) 5.0+
  • Can use Aerospike Security accounts

Requirements

Additional python modules are required and installed using pip:

sudo pip install -r requirements.txt

See requirements.txt

Install

  1. Clone the repository and place aerospike_types.db, aerospike_schema.yaml and aerospike_plugin.py somewhere accessible by collectd.
  • This is /opt/collectd-plugins for all 3 files files in the example below.
  • Ensure that aerospike_plugin.py is executable. (ie: chmod +x aerospike_plugin.py)
  1. Drop aerospike.conf into the collectd.conf.d directory (typically found at /etc/collectd/collectd.conf.d) or copy its contents into /etc/collectd/collectd.conf
  2. Reload/restart collectd
  3. Check that collectd is working fine:
tail -f /var/log/syslog
May 28 00:27:56 host1 collectd[21621]: Aerospike Plugin: client 127.0.0.1:3000
May 28 00:27:56 host1 collectd[21621]: Aerospike Plugin: Counter({'writes': 262, 'emits': 318})
May 28 00:28:06 host1 collectd[21621]: Aerospike Plugin: client 127.0.0.1:3000
May 28 00:28:06 host1 collectd[21621]: Aerospike Plugin: Counter({'writes': 262, 'emits': 318})

Note: If you run into errors about types.db not defined:

collectd[21431]: plugin_dispatch_values: Dataset not found: memory (from "host1/memory/memory-used"), check your types.db!

Edit the /etc/collectd/collectd.conf file and explicitly add (uncomment) the default types:

TypesDB "/usr/share/collectd/types.db"

Highlights from collectd.conf:

TypesDB  "/usr/share/collectd/types.db" "/opt/collectd-plugins/aerospike_types.db"

<LoadPlugin python>
    Globals true
</LoadPlugin>

<Plugin python>
    ModulePath "/opt/collectd-plugins/"
    LogTraces true
    Interactive false
    Import "aerospike_plugin"
    <Module aerospike_plugin>
        Host   "127.0.0.1"
        Port   3000
        # Prefix "cluster_name"
        # HostNameOverride "clusters.cluster_name.host_name"
    </Module>
</Plugin>

Authentication Support

If Aerospike is configured with authentication, then you will need to configure the plugin to authenticate.

To configure the username and password for authenticating the plugin, specify User and Password in the configuration as follows. AuthMode is optional parameter to specify authentication mode. It's default value is INTERNAL.

<Plugin python>
    ModulePath "/opt/collectd-plugins/"
    LogTraces true
    Interactive false
    Import "aerospike_plugin"
    <Module aerospike_plugin>
    	User "admin"
    	Password "admin"
    	AuthMode "EXTERNAL"
    </Module>
</Plugin>

SSL/TLS Support

If Aerospike is configured with SSL/TLS, then you will need to configure the plugin with SSL/TLS as well.

SSL/TLS parameters are as follows:

<Plugin python>
    ModulePath "/opt/collectd-plugins/"
    Import "aerospike_plugin"

    <Module aerospike_plugin>

        Port "4333"
        TLSEnable true
        TLSName "my.aerospike.server"
#       TLSKeyfile ""
#       TLSKeyfilePw ""
#       TLSCertfile ""
        TLSCAFile "/etc/ssl/rootCA.pem"
#       TLSCAPath ""
#       TLSCipher "ALL"
#       TLSProtocols "all"
#       TLSBlacklist ""
#       TLSCRLCheck true
#       TLSCRLCheckAll true

    </Module>
</Plugin>
  • Port - The secured port that Aerospike is listening on.
  • TLSEnable - Required for TLS. Enable TLS plugin. Default False.
  • TLSName - The hostname on the server's certificate. Required for normal auth and mutual auth.
  • TLSKeyfile - The private key for your client cert. Required for mutual auth.
  • TLSKeyfilePw - Decryption password for the private key. By default the key is assumed not to be encrypted.
  • TLSCertfile - The certificate for your client. Required for mutual auth.
  • TLSCAFile - Required The CA root certificate.
  • TLSCAPath - The path to CAs and/or Certificate Revocation Lists.
  • TLSCipher - The TLS Ciphers to use. See https://www.openssl.org/docs/man1.1.0/man1/ciphers.html for list of available ciphers. Must agree with server.
  • TLSProtocols - The SSL/TLS protocols to use.
  • TLSBlacklist - A file containing the serial numbers of blacklisted certificates.
  • TLSCRLCheck - Check against leaf certificates in the CRL chain.
  • TLSCRLCheckAll - Check against all certificates in the CRL chain.

SSL/TLS Protocols

Available protocols are: TLSv1, TLSv1.1, TLSv1.2

To use any supported protocol, a special keyword all may be used.

You can also include individual protocols by prepending a +, eg: +TLSv1.1.
You can also exclude individual protocols by prepending a -, eg: -TLSv1.

Examples

To use Server Side Authentication mode:

    <Module aerospike_plugin>
        Port "4333"
        TLSEnable true
        TLSName "my.aerospike.server"
        TLSCAFile "/etc/ssl/rootCA.pem" # Optional. Required for self-signed certs
    </Module>

To use Mutual Authentication mode:

    <Module aerospike_plugin>
        Port "4333"
        TLSEnable true
        TLSName "my.aerospike.server"
        TLSKeyfile "/etc/ssl/my.key"
        TLSCertfile "/etc/ssl/cert.pem"
        TLSCAFile "/etc/ssl/rootCA.pem"
    </Module>

aerospike-collectd's People

Contributors

cstivers78 avatar jdogmcsteezy avatar kportertx avatar mtendjou avatar rbotzer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aerospike-collectd's Issues

Parse error in file aerospike_types.db

I tried to set up aerospike-collectd plugin on the collectd which has provided by google stackdriver. I've got some errors as following. Would you mind giving me advise to address it?

error logs:

Sep 30 05:26:09 host-2 stackdriver-agent[12626]: mesg: ttyname failed: Inappropriate ioctl for device
Sep 30 05:26:09 host-2 stackdriver-agent[12626]:  * Stopping Stackdriver metrics collection agent stackdriver-agent
Sep 30 05:26:12 host-2 stackdriver-agent[12626]:  .   ...done.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: mesg: ttyname failed: Inappropriate ioctl for device
Sep 30 05:26:12 host-2 stackdriver-agent[12651]:  * Starting Stackdriver metrics collection agent stackdriver-agent
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_plugin.py', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_plugin.py'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_schema.yaml', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_schema.yaml'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_types.db', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_types.db'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = Interval; value = 60.000000;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = Hostname; value = ;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = FQDNLookup; value = false;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Created new plugin context.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: ====''=..()=..()=..(,'.')='-option = PIDFile; value = /var/run/stackdriver-agent.pid;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_plugin.py', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_plugin.py'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_schema.yaml', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_schema.yaml'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Parse error in file `/opt/collectd-plugins/aerospike_types.db', line 165 near `/': syntax error, unexpected SLASH
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: yyparse returned error #1
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: configfile: Cannot read file `/opt/collectd-plugins/aerospike_types.db'.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = Interval; value = 60.000000;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = Hostname; value = ;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: option = FQDNLookup; value = false;
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: Created new plugin context.
Sep 30 05:26:12 host-2 stackdriver-agent[12651]: ====''=..()=..()=..(,'.')='-   ...done.

env info:

Not working on Python3 and Ubuntu 20

Hello there, I was unable to make it running on Ubuntu 20 LTS with python3 installed and all necessary modules.
With python2 and pip build fails, and the first thing I had to do is to change to aerospike-client version to 4.0 in requirements.txt, and additionally, I had to fix importing of ServerError exception inside of aerospike_plugin.py.
However, I am still unable to run it successfully. Could you please help somehow?

Here is the last log.
aerospike_log.txt

Syntax error

aerospike_plugin.py", line 394
if any(m == name) for m in metrics):
^
SyntaxError: invalid syntax

python-2.7.5-34.el7.x86_64

Can't get read and write reqs counters

I am using aerospike-collectd v1.0.3 plugin with Aerospike Server Community Edition v3.9.1.1 (Linux Debian Jessie)

The plugin works well for some counters like aerospike-service.gauge-client_connections and some other counters.

But I cannot retrieve the read and write counters for a while: I do not get any value for aerospike-service.operations-stat_write_reqs and aerospike-service.operations-stat_read_reqs.

These counters used to work a few months ago with an Aerospike v3.8 server but it does not work anymore since an Aerospike server upgrade (I think).

Is it supposed to work on Aerospike 3.9.1.1? Or did I miss something?

If TLSEnable is false, ssl_context is not initialized

[2017-02-02 11:41:57] Unhandled python exception in read callback: UnboundLocalError: local variable 'ssl_context' referenced before assignment
[2017-02-02 11:41:57] Traceback (most recent call last):
[2017-02-02 11:41:57]   File "/opt/collectd-aerospike/aerospike_plugin.py", line 755, in read
    capath=ca_path, protocols=protocols, cert_blacklist=blacklist, crl_check=crl_check, crl_check_all=crl_check_all, tls_name=tls_name)
[2017-02-02 11:41:57]   File "/opt/collectd-aerospike/aerospike_plugin.py", line 230, in connect

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.