Giter Club home page Giter Club logo

kafka's Introduction

Kafka

Master: Build Status Develop: Build Status

This roles installs Apache Kafka server.

For more information about Kafka please visit zookeeper.apache.org/.

Installation and Dependencies

This role will install sansible.users_and_groups for managing kafka user.

To install run ansible-galaxy install sansible.kafka or add this to your roles.yml

- name: sansible.kafka
  version: v2.0

and run ansible-galaxy install -p ./roles -r roles.yml

AWS Setup

This role has AWS support built in for deployment/discovery.

AWS Tag Discovery

Designed for instances that are statically defined either as direct EC2 instances or via a single ASG per instance.

The broker.id is derived from a tag attached to the instance, you can turn on this behaviour and specify the tag to lookup like so:

- role: sansible.kafka
  sansible_kafka_aws_cluster_assigned_id_enabled: yes
  sansible_kafka_aws_cluster_assigned_id_tag_name: instanceindex
  # A ZK cluster behind an ELB
  sansible_kafka_zookeeper_hosts:
    - zookeeper.app.internal

Tags

This role uses two tags: build and configure

  • build - Installs Kafka server and all its dependencies.
  • configure - Configure and ensures that the Kafka service is running.

Examples

- name: Install Kafka Server
  hosts: sandbox

  pre_tasks:
    - name: Update apt
      become: yes
      apt:
        cache_valid_time: 1800
        update_cache: yes
      tags:
        - build

  roles:
    - name: sansible.kafka
      sansible_kafka_zookeeper_hosts:
        - my.zookeeper.host
- name: Install Kafka with NewRelic integration
  hosts: sandbox

  pre_tasks:
    - name: Update apt
      become: yes
      apt:
        cache_valid_time: 1800
        update_cache: yes
      tags:
        - build

  roles:
    - name: sansible.kafka
      sansible_kafka_environment_vars:
        - "NEWRELIC_OPTS=\"-javaagent:/home/{{ sansible_kafka_user }}/newrelic/newrelic.jar\""
        - "export KAFKA_OPTS=\"${KAFKA_OPTS} ${NEWRELIC_OPTS}\""

If you just want to test Kafka service build both Zookeeper and Kafka on the same machine.

- name: Install Kafka Server
  hosts: sandbox

  pre_tasks:
    - name: Update apt
      become: yes
      apt:
        cache_valid_time: 1800
        update_cache: yes
      tags:
        - build

  roles:
    - name: sansible.zookeeper
    - name: sansible.kafka

Logging access denied. These access logs will be placed in {{ sansible_kafka_log_dir }}/kafka-authorizer.log.

- role: sansible.kafka
sansible_kafka_authorizer_log_level: INFO

kafka's People

Contributors

dennisconrad avatar dm avatar jbrissier avatar js-kingkong-bot avatar lobsterdore avatar mehonoshin avatar msahota14 avatar quater avatar racer1680 avatar rsgoodall avatar seliverstov avatar timgrist avatar wojtek-oledzki avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.