Giter Club home page Giter Club logo

docker-efk's Introduction

ElasticSearch Docker

This repo demonstrates some usage of ElasticSearch in Docker.

Start

$ docker-compose up -d

Verify

$ docker-compose ps

Output:

            Name                           Command               State                         Ports
---------------------------------------------------------------------------------------------------------------------------
elasticsearch_elasticsearch_1   /usr/local/bin/docker-entr ...   Up      127.0.0.1:9200->9200/tcp, 127.0.0.1:9300->9300/tcp

Check Health

$ curl localhost:9200/_cat/health

Output:

1522654718 07:38:38 docker-cluster green 1 1 1 1 0 0 0 0 - 100.0%

Create Index

$ curl -X PUT 127.0.0.1:9200/schools

Output:

{"acknowledged":true,"shards_acknowledged":true,"index":"schools"}

UI

Enter the elasticsearch url http://127.0.0.1:9200 at the following url:

$ open http://127.0.0.1:1358

webui

Post Data

$ curl -XPOST -H "Content-Type:application/x-ndjson" --data-binary @data.json http://127.0.0.1:9200/schools/_bulk

Output:

{"took":168,"errors":false,"items":[{"index":{"_index":"schools","_type":"school","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"schools","_type":"school","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"schools","_type":"school","_id":"3","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}}]}

Search

Search all:

$ curl -XPOST -H "Content-Type:application/json" -d '{}' http://127.0.0.1:9200/schools/_search

Output:

{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":3,"max_score":1.0,"hits":[{"_index":"schools","_type":"school","_id":"2","_score":1.0,"_source":{"name":"Saint Paul School", "description":"ICSE Afiliation", "street":"Dawarka", "city":"Delhi", "state":"Delhi", "zip":"110075", "location":[28.5733056, 77.0122136], "fees":5000, "tags":["Good Faculty", "Great Sports"], "rating":"4.5"}},{"_index":"schools","_type":"school","_id":"1","_score":1.0,"_source":{"name":"Central School", "description":"CBSE Affiliation", "street":"Nagan","city":"paprola", "state":"HP", "zip":"176115", "location":[31.8955385, 76.8380405],"fees":2000, "tags":["Senior Secondary", "beautiful campus"], "rating":"3.5"}},{"_index":"schools","_type":"school","_id":"3","_score":1.0,"_source":{"name":"Crescent School", "description":"State Board Affiliation", "street":"Tonk Road", "city":"Jaipur", "state":"RJ", "zip":"176114","location":[26.8535922, 75.7923988],"fees":2500, "tags":["Well equipped labs"], "rating":"4.5"}}]}}

Search specific query:

$ curl -XPOST -H "Content-Type:application/json" -d @search.json http://127.0.0.1:9200/schools/_search

Output:

{"took":7,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0.2876821,"hits":[{"_index":"schools","_type":"school","_id":"1","_score":0.2876821,"_source":{"name":"Central School", "description":"CBSE Affiliation", "street":"Nagan","city":"paprola", "state":"HP", "zip":"176115", "location":[31.8955385, 76.8380405],"fees":2000, "tags":["Senior Secondary", "beautiful campus"], "rating":"3.5"}}]}}

View Indices

$ curl http://127.0.0.1:9200/_cat/indices?v

Kibana

Kibana is available at http://127.0.0.1:5601.

kibana

Fluentd

$ docker pull ubuntu

$ docker run -t -i --log-driver=fluentd ubuntu echo "Hello Fluent Bit!"
$ docker run --rm --log-driver=fluentd ubuntu /bin/echo 'Hello world'

Trigger logs:

$ repeat 10 curl http://localhost:80/

kibana-fluentd

TODO

  • Add example on how to sync data from mysql to elasticsearch
  • Add example on how to sync data from elasticsearch to S3
  • See how to use elasticsearch as complete logging tools for microservices
  • See if it is possible to integrate grafana or prometheus
  • Setup it in Kubernetes and Nomad environment

docker-efk's People

Contributors

alextanhongpin avatar joaopluigi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.