Giter Club home page Giter Club logo

ansible-role-elk's Introduction

ELK Ansible role

Installs the ELK stack (Elasticsearch, Logstash, and Kibana) for log aggregation and monitoring. Intended for integration with Riemann for alerting functionality.

Requirements

Role variables

elk_kibana_user: "kibana"
elk_kibana_logfile: "/var/log/kibana.log"

# Provide ability to disable the snapshot functionality. It's not well
# tested, so leaving false as the default now. If set to true on a first
# run, probably should add `meta: flush_handlers` prior to running to ensure
# the `path.repo:` variable is recognized by the running elasticsearch service..
elk_elasticsearch_snapshot: false

elk_elasticsearch_snapshot_directory: /var/lib/elasticsearch/backups
elk_elasticsearch_snapshot_repository: es_backup
elk_elasticsearch_snapshot_initialization:
  type: fs
  settings:
    location: "{{ elk_elasticsearch_snapshot_directory }}"
    compress: yes
  _hack: null

# It'd be nice to use iso8601 instead epoch, but the ElasticSearch API
# throws an invalid_snapshot_name error with the iso8601 format.
elk_elasticsearch_snapshot_name: "snapshot-{{ ansible_date_time.epoch }}"

# Limits to set in /etc/security/limits.conf. Make sure to copy the entire
# list if overriding any of the individual elements.
elk_elasticsearch_pam_limits:
    - domain: elasticsearch
      limit_item: memlock
      limit_type: hard
      value: unlimited

    - domain: elasticsearch
      limit_item: memlock
      limit_type: soft
      value: unlimited

    - domain: elasticsearch
      limit_item: nofile
      limit_type: soft
      value: 65535

    - domain: elasticsearch
      limit_item: nofile
      limit_type: hard
      value: 65535

# Riemann plugin for alerting, de-dot filter for ElasticSearch v2 compatibility.
# See: https://www.elastic.co/blog/introducing-the-de_dot-filter
elk_logstash_plugins:
  - logstash-output-riemann
  - logstash-filter-de_dot

# Interface used for firewall restrictions and IPv4 lookups
elk_network_interface: eth0

elk_cluster_name: elk-logging

# SSL is disabled by default. Set these vars to the fullpaths to SSL
# certs you wish to use, and Nginx will force HTTPS connections.
# You must place the SSL certs there in a separate play.
elk_nginx_ssl_certificate: ""
elk_nginx_ssl_certificate_key: ""
elk_nginx_server_name: localhost

# Not safe for production use! Override to secure logins.
elk_kibana_username: kibana
elk_kibana_password: kibana

# Override to change the landing page, e.g. a custom dashboard:
# "dashboard/Your-Dashboard-Name". You must replace whitespace in
# dashboard names with hyphens, since Kibana expects it.
elk_kibana_default_app: discover

# Enable automatic configuration of IP whitelisting for "logclients".
# Uses ufw. Disable if you're using a different role for firewall config.
elk_configure_firewall: true

# Allow downstream playbooks to utilize custom webserver configuration
# Set this to false in order to skip over this role's nginx rollout
elk_configure_nginx: true

# Allow downstream playbooks to override patterns and filters fileglob list
elk_logstash_patterns:
  - logstash-patterns/*
elk_logstash_filters:
  - logstash-configs/*

# Declare fileglob of GeoIP databases to copy. Off by default.
elk_logstash_geoipdbs: []

Usage

Use the role in a playbook like this:

- hosts: logserver
  roles:
    - role: elk
      elk_kibana_username: admin
      elk_kibana_password: WowWhatAStrongPassword4

Adding visualizations

The role does not yet create Kibana visualizations automatically. You can however import saved visualizations that ship with the role in files/kibana-dashboards.json. Go to Settings -> Objects -> Import in the Kibana UI and browse to the JSON file.

Running the tests

This role uses Molecule and ServerSpec for testing. To use it:

pip install molecule
gem install serverspec
molecule test

You can also run selective commands:

molecule idempotence
molecule verify

See the Molecule docs for more info.

Further reading

Setup

Developing custom filters

See the examples/writing-filters directory in this repo for a preconfigured development environment. Copy that directory to a server with logstash installed, or use a Vagrant testing VM.

Maintenance

License

MIT

ansible-role-elk's People

Contributors

ageis avatar conorsch avatar msheiny avatar techraf avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.