Giter Club home page Giter Club logo

elkx-docker's Introduction

Elasticsearch, Logstash, Kibana, X-Pack (ELKX) Docker image

This Docker image provides a convenient centralised log server and log management web interface, by packaging Elasticsearch, Logstash, and Kibana, collectively known as ELK, and extends this stack with X-Pack, which bundles security, alerting, monitoring, reporting, and graph capabilities.

This image is hosted on Docker Hub at https://hub.docker.com/r/sebp/elkx/.

This image is frozen as of version 6.2.4 of the ELK stack. Since version 6.3, X-Pack is installed with Elasticsearch, Logstash, and Kibana, making this image somewhat irrelevant. In order to help users set up X-Pack, X-Pack-specific helper scripts and documentation that were developed for the sebp/elkx image will be copied over to the repository for the sebp/elk image for guidance purposes, but it will be up to the users to actually enable and configure X-Pack themselves using Elastic's official documentation if they need the corresponding features.

The following tags are available:

  • latest, 624: ELKX 6.2.4.

  • 623: ELKX 6.2.3.

  • 622: ELKX 6.2.2.

  • 621: ELKX 6.2.1.

  • 620: ELKX 6.2.0.

  • 613: ELKX 6.1.3.

  • 612: ELKX 6.1.2.

  • 611: ELKX 6.1.1.

  • 610: ELKX 6.1.0.

  • 601: ELKX 6.0.1.

  • 600: ELKX 6.0.0.

  • 562: ELKX 5.6.2.

  • 561: ELKX 5.6.1.

  • 560: ELKX 5.6.0.

  • 553: ELKX 5.5.3.

  • 552: ELKX 5.5.2.

  • 551: ELKX 5.5.1.

  • 550: ELKX 5.5.0.

  • 543: ELKX 5.4.3.

  • 542: ELKX 5.4.2.

  • 541: ELKX 5.4.1.

  • 540: ELKX 5.4.0.

  • 532: ELKX 5.3.2.

  • 531: ELKX 5.3.1.

  • 530: ELKX 5.3.0.

  • 522: ELKX 5.2.2.

  • 521: ELKX 5.2.1.

  • 520: ELKX 5.2.0.

  • 512: ELKX 5.1.2.

  • 511: ELKX 5.1.1.

  • 502: ELKX 5.0.2.

Quickstart using Docker Compose

Create the following docker-compose.yml file.

elkx:
  image: sebp/elkx
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"
  environment:
    - ELASTIC_BOOTSTRAP_PASSWORD="changeme"

Start a container using docker-compose.

$ docker-compose up
Creating elkxdocker_elkx_1
Attaching to elkxdocker_elkx_1
elkx_1  | ERROR: Setting [bootstrap.pass] does not exist in the keystore.
elkx_1  |  * Starting periodic command scheduler cron
elkx_1  |    ...done.
elkx_1  |  * Starting Elasticsearch Server
elkx_1  |    ...done.
elkx_1  | waiting for Elasticsearch to be up (1/30)
...

In another shell, open a bash session in the running container (replacing <name of the running container> with the right value), and use X-Pack's setup-passwords tool (located in $ES_HOME/bin/x-pack) to set the passwords for the built-in users.

$ docker exec -it <name of the running container> bash
# $ES_HOME/bin/x-pack/setup-passwords interactive
Initiating the setup of reserved user elastic,kibana,logstash_system passwords.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: changeme
Reenter password for [elastic]: changeme
Enter password for [kibana]: changeme
Reenter password for [kibana]: changeme
Enter password for [logstash_system]: changeme
Reenter password for [logstash_system]: changeme
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [elastic]

Stop the container, then edit the docker-compose.yml as follows:

elkx:
  image: sebp/elkx
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"
  environment:
    - ELASTICSEARCH_USER=elastic
    - ELASTICSEARCH_PASSWORD=changeme
    - LOGSTASH_USER=elastic
    - LOGSTASH_PASSWORD=changeme
    - KIBANA_USER=kibana
    - KIBANA_PASSWORD=changeme

Then start the container again using docker-compose up.

Usage notes

This image extends the sebp/elk image, so unless otherwise noted below the documentation for the seb/elk image applies.

Bootstrap mode

This image uses the default configuration of X-Pack, meaning that out of the box, as from version 6, the built-in users (especially the elastic superuser, and the basic kibana user) no longer have default passwords.

To define passwords (and create additional users as needed), a container first needs to be started in bootstrap mode in order to assign a bootstrap password to the elastic superuser.

As described in the official X-Pack documentation:

The bootstrap password is a transient password that enables you to run the tools that set all the built-in user passwords.

To set the bootstrap password for elastic, start a container with the ELASTIC_BOOTSTRAP_PASSWORD environment variable set to the chosen password.

Once the container has started, only Elasticsearch will be running, and the user can use the elastic account (with the bootstrap password) to change its own password and assign passwords to the built-in users, for instance:

Once all the passwords have been assigned, stop the container, and start the container in normal mode as described below.

Running the container in normal mode

In order to start up and run normally, the container needs to have two users that are authorised to connect to Elasticsearch's and Kibana's interfaces (JSON and web, respectively), and their credentials must be set using the following environment variables: ELASTICSEARCH_USER, ELASTICSEARCH_PASSWORD, KIBANA_USER, and KIBANA_PASSWORD.

In addition, the default Logstash configuration (in /etc/logstash/conf.d/30-output.conf) uses the user defined by the LOGSTASH_USER and LOGSTASH_PASSWORD environment variables to sends logs to Elasticsearch.

To get an idea of how this works, in a non-production environment, first set passwords for the built-in elastic and kibana users to changeme in bootstrap mode as described above, then re-run the container with:

  • ELASTICSEARCH_USER and LOGSTASH_USER both set to elastic (i.e. we'll be using the built-in superuser to monitor Elasticsearch and send it logs from Logstash),

  • KIBANA_USER set to kibana,

  • ELASTICSEARCH_PASSWORD, LOGSTASH_PASSWORD, and KIBANA_PASSWORD all set to changeme.

Creating a dummy log entry

Building on the previous example, in order to create a dummy log entry in Elasticsearch using the elastic superuser account, docker exec inside the running container (see the Creating a dummy log entry section of the ELK Docker image documentation), and use the following command instead of the original one (replace the password with the one you set for the elastic user):

# /opt/logstash/bin/logstash --path.data /tmp/logstash/data \
	-e 'input { stdin { } } output { elasticsearch { hosts => ["localhost"] user => "elastic" password => "changeme" } }'

This entry can then be viewed by logging into Kibana as elastic.

Forwarding logs with Filebeat: example set-up and configuration

To run the example Filebeat set-up with ELKX, use the nginx-filebeat subdirectory of the source Git repository on GitHub, and update the credentials to connect to Elasticsearch in start.sh before building the image.

Development mode

The latest image includes a development mode, which disables X-Pack security in Elasticsearch and Kibana, thereby eliminating the need to set up user credentials as described above.

To start a container in development mode, set the DEVELOPMENT_MODE environment variable to 1.

Security considerations

X-Pack allows for a secured set-up of the ELK stack, but by default this image is insecure (no message authentication, no auditing, default certificates).

See the X-Pack documentation on Getting Started with Security for guidance on how to secure ELK with X-Pack.

About

Written by Sébastien Pujadas, released under the Apache 2 license.

elkx-docker's People

Contributors

cappadona avatar kenwdelong avatar nonotest avatar spujadas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elkx-docker's Issues

Kibana doesn't show up

Hi, I played with your ELK image for somme days before decided to switch to this one to add a security layer to my diggings.

After some tests, Kibana doesn't start and it's log file is empty.

elk:
  image: sebp/elkx
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"
  environment:
    TZ: 'Europe/Paris'
  volumes:
    - ./etc/kibana.yml:/opt/kibana/config/kibana.yml
    - ./etc/logstash/40-srcds.conf:/etc/logstash/conf.d/40-srcds.conf
    - ./patterns/srcds:/opt/logstash/patterns/srcds
    - ./data:/var/lib/elasticsearch
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::9200                 :::*                    LISTEN      -
tcp6       0      0 :::5044                 :::*                    LISTEN      -
tcp6       0      0 :::9300                 :::*                    LISTEN      -
tcp6       0      0 127.0.0.1:9600          :::*                    LISTEN      -
 docker exec -it elkx_elk_1 ls -la /var/log/kibana/kibana5.log
-rw-r--r-- 1 kibana kibana 0 Apr 12 14:21 /var/log/kibana/kibana5.log
docker logs elkx_elk_1
 * Starting periodic command scheduler cron
   ...done.
 * Starting Elasticsearch Server
   ...done.
waiting for Elasticsearch to be up (1/30)
waiting for Elasticsearch to be up (2/30)
waiting for Elasticsearch to be up (3/30)
waiting for Elasticsearch to be up (4/30)
waiting for Elasticsearch to be up (5/30)
waiting for Elasticsearch to be up (6/30)
waiting for Elasticsearch to be up (7/30)
waiting for Elasticsearch to be up (8/30)
waiting for Elasticsearch to be up (9/30)
Waiting for Elasticsearch cluster to respond (1/30)
logstash started.
 * Starting Kibana5
   ...done.
==> /var/log/elasticsearch/elasticsearch.log <==
[2017-04-12T14:21:10,088][INFO ][o.e.n.Node               ] [WNRkAuz] starting ...
[2017-04-12T14:21:10,266][WARN ][i.n.u.i.MacAddressUtil   ] Failed to find a usable hardware address from the network interfaces; using random bytes: 54:80:85:7a:92:8e:9d:5a
[2017-04-12T14:21:10,345][INFO ][o.e.t.TransportService   ] [WNRkAuz] publish_address {172.17.0.2:9300}, bound_addresses {[::]:9300}
[2017-04-12T14:21:10,356][INFO ][o.e.b.BootstrapChecks    ] [WNRkAuz] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-04-12T14:21:13,510][INFO ][o.e.c.s.ClusterService   ] [WNRkAuz] new_master {WNRkAuz}{WNRkAuzoR3-Zi3BB2ZvrFw}{JFGuHAGtSMeNtpuy8P2T6w}{172.17.0.2}{172.17.0.2:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-12T14:21:13,551][INFO ][o.e.h.HttpServer         ] [WNRkAuz] publish_address {172.17.0.2:9200}, bound_addresses {[::]:9200}
[2017-04-12T14:21:13,551][INFO ][o.e.n.Node               ] [WNRkAuz] started
[2017-04-12T14:21:14,157][INFO ][o.e.l.LicenseService     ] [WNRkAuz] license [3407e1bb-2b66-4638-9d2b-90bf132b9bb8] mode [trial] - valid
[2017-04-12T14:21:14,178][INFO ][o.e.g.GatewayService     ] [WNRkAuz] recovered [3] indices into cluster_state
[2017-04-12T14:21:14,944][INFO ][o.e.c.r.a.AllocationService] [WNRkAuz] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.monitoring-es-2-2017.04.12][0], [.monitoring-data-2][0]] ...]).

==> /var/log/logstash/logstash-plain.log <==

==> /var/log/kibana/kibana5.log <==

==> /var/log/logstash/logstash-plain.log <==
[2017-04-12T14:21:33,987][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/opt/logstash/data/queue"}
[2017-04-12T14:21:34,013][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"f5184e09-b481-48d9-a3d4-48c695f0078d", :path=>"/opt/logstash/data/uuid"}
[2017-04-12T14:21:35,161][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@localhost:9200/]}}
[2017-04-12T14:21:35,164][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://elastic:xxxxxx@localhost:9200/, :path=>"/"}
[2017-04-12T14:21:35,367][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>#<URI::HTTP:0x4e948282 URL:http://elastic:xxxxxx@localhost:9200/>}
[2017-04-12T14:21:35,375][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>[#<URI::Generic:0x7990f6ba URL://localhost>]}
[2017-04-12T14:21:35,701][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>250}
[2017-04-12T14:21:36,134][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2017-04-12T14:21:36,190][INFO ][logstash.pipeline        ] Pipeline main started
[2017-04-12T14:21:36,255][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Index not appearing in Kibana

But while checking with curl, there are clearly documents of that index in elasticsearch.

However Kibana keeps complainging that "Unable to fetch mapping. Do you have indices matching the pattern?"

Support disable x-pack security

Hi,

Would you please add env variable so as to disable the security for the purpose of dev usage? I believe it will be much convenient.

Thanks!

Adding credentials...

With regards to changing the passwords for the primary elastic user.

Could we simply add something like this to the dockerfile? In the correct format?

RUN curl -XPUT -u elastic 'localhost:9200/_xpack/security/user/elastic/_password' -H "Content-Type: application/json" -d '{
"password" : "elasticpassword"
}'

I know you mentioned env variables as an option.

Ideally i'm trying to change the password and create a read only user during the build process.

Any thoughts on how to get started would be VERY appreciated. Thanks for the wonderful image.

Best,
Shawn

Image at Docker Hub out of date?

I am new to docker and ELK so it might be a mistake on my side but it looks like the images on Docker Hub are out of date:

sebp/elkx pulls version 6.2.4

sebp/elkx:632 is unavailable:

Pulling elkx (sebp/elkx:632)...
ERROR: manifest for sebp/elkx:632 not found

Enable X-Pack security for Kibana only

Do you think is it possible to enable X-Pack security for Kibana only?

I have use case in which Elasticsearch & Logstash will be used internally in a private network and not open to the internet whereas Kibana will be available at https://logs.example.com.

Looking and having tried a little with the current configuration, I think, it requires a fair bit amount of overriding to Dockerfile and/or startx.sh.

Do you think something similar to Starting services selectively in https://github.com/spujadas/elk-docker could be implemented, such that one could do (for example) ELASTICSEARCH_USE_XPACK_SECURITY=0 (defaults to ELASTICSEARCH_USE_XPACK_SECURITY=1) to disable X-Pack security for Elasticsearch (and Logstash)?

Unable to fetch mapping

Hello, i triied elk image for some days now. I then switched on your elkx image but i can't configure an index pattern, it keeps telling me that no indice is matching the pattern, while there is in fact.

Seems like SIGTERM trap doesn’t work

When shutting down a container (i.e. CTRL-c) in https://github.com/spujadas/elk-docker, I can see…

_term() {
  echo "Terminating ELK"
  service elasticsearch stop
  service logstash stop
  service kibana stop
  exit 0
}

… executes, outputs “Terminating ELK” and stops services correctly.

This doesn’t seem to work in elkx-docker (here) however.

Maybe it is because start.sh (CMD for elk-docker image) executed internally from startx.sh (CMD for elkx-docker image)?

Thanks.

container doesn t start

Hi,

I have tested all your troubleshouting techniks and I can t figure out what s wrong.

I try to run under rancher this compose:

elkx:
  ports:
  - 5044:5044/tcp
  - 5601:5601/tcp
  - 9200:9200/tcp
  labels:
    io.rancher.scheduler.affinity:host_label: dwl=dwlComPublic
  image: sebp/elkx:520
  volumes:
  - elkx:/var/lib/elasticsearch

and I got these logs


3/21/2017 12:22:19 PM * Starting periodic command scheduler cron
3/21/2017 12:22:19 PM   ...done.
3/21/2017 12:22:19 PM * Starting Elasticsearch Server
3/21/2017 12:22:34 PM   ...fail!
3/21/2017 12:22:37 PMwaiting for Elasticsearch to be up (1/30)
3/21/2017 12:22:38 PMwaiting for Elasticsearch to be up (2/30)
3/21/2017 12:22:39 PMwaiting for Elasticsearch to be up (3/30)
3/21/2017 12:22:40 PMwaiting for Elasticsearch to be up (4/30)
3/21/2017 12:22:41 PMwaiting for Elasticsearch to be up (5/30)
3/21/2017 12:22:43 PMwaiting for Elasticsearch to be up (6/30)
3/21/2017 12:22:44 PMwaiting for Elasticsearch to be up (7/30)
3/21/2017 12:22:45 PMwaiting for Elasticsearch to be up (8/30)
3/21/2017 12:22:46 PMwaiting for Elasticsearch to be up (9/30)
3/21/2017 12:22:47 PMwaiting for Elasticsearch to be up (10/30)
3/21/2017 12:22:48 PMwaiting for Elasticsearch to be up (11/30)
3/21/2017 12:22:49 PMwaiting for Elasticsearch to be up (12/30)
3/21/2017 12:22:50 PMwaiting for Elasticsearch to be up (13/30)
3/21/2017 12:22:51 PMwaiting for Elasticsearch to be up (14/30)
3/21/2017 12:22:52 PMwaiting for Elasticsearch to be up (15/30)
3/21/2017 12:22:53 PMwaiting for Elasticsearch to be up (16/30)
3/21/2017 12:22:54 PMwaiting for Elasticsearch to be up (17/30)
3/21/2017 12:22:55 PMwaiting for Elasticsearch to be up (18/30)
3/21/2017 12:22:56 PMwaiting for Elasticsearch to be up (19/30)
3/21/2017 12:22:57 PMwaiting for Elasticsearch to be up (20/30)
3/21/2017 12:22:59 PMwaiting for Elasticsearch to be up (21/30)
3/21/2017 12:23:00 PMwaiting for Elasticsearch to be up (22/30)
3/21/2017 12:23:01 PMwaiting for Elasticsearch to be up (23/30)
3/21/2017 12:23:02 PMwaiting for Elasticsearch to be up (24/30)
3/21/2017 12:23:03 PMwaiting for Elasticsearch to be up (25/30)
3/21/2017 12:23:04 PMwaiting for Elasticsearch to be up (26/30)
3/21/2017 12:23:05 PMwaiting for Elasticsearch to be up (27/30)
3/21/2017 12:23:07 PMwaiting for Elasticsearch to be up (28/30)
3/21/2017 12:23:08 PMwaiting for Elasticsearch to be up (29/30)
3/21/2017 12:23:09 PMwaiting for Elasticsearch to be up (30/30)
3/21/2017 12:23:09 PMCouln't start Elasticsearch. Exiting.
3/21/2017 12:23:09 PMElasticsearch log follows below.
3/21/2017 12:23:09 PM[2017-03-21T11:19:05,602][INFO ][o.e.n.Node               ] [] initializing ...
3/21/2017 12:23:09 PM[2017-03-21T11:19:06,240][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/sda2)]], net usable_space [374.4gb], net total_space [457.2gb], spins? [possibly], types [ext4]
3/21/2017 12:23:09 PM[2017-03-21T11:19:06,241][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] heap size [1.9gb], compressed ordinary object pointers [true]
3/21/2017 12:23:09 PM[2017-03-21T11:19:06,418][INFO ][o.e.n.Node               ] node name [ZVc9ZHX] derived from node ID [ZVc9ZHX3RROOSsuo2ebnaw]; set [node.name] to override
3/21/2017 12:23:09 PM[2017-03-21T11:19:06,425][INFO ][o.e.n.Node               ] version[5.2.0], pid[48], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/3.13.0-93-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_121/25.121-b13]
3/21/2017 12:23:09 PM[2017-03-21T11:20:17,764][INFO ][o.e.n.Node               ] [] initializing ...
3/21/2017 12:23:09 PM[2017-03-21T11:20:21,448][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/sda2)]], net usable_space [374.4gb], net total_space [457.2gb], spins? [possibly], types [ext4]
3/21/2017 12:23:09 PM[2017-03-21T11:20:21,448][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] heap size [1.9gb], compressed ordinary object pointers [true]
3/21/2017 12:23:09 PM[2017-03-21T11:20:21,578][INFO ][o.e.n.Node               ] node name [ZVc9ZHX] derived from node ID [ZVc9ZHX3RROOSsuo2ebnaw]; set [node.name] to override
3/21/2017 12:23:09 PM[2017-03-21T11:20:21,585][INFO ][o.e.n.Node               ] version[5.2.0], pid[46], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/3.13.0-93-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_121/25.121-b13]
3/21/2017 12:23:09 PM[2017-03-21T11:21:30,390][INFO ][o.e.n.Node               ] [] initializing ...
3/21/2017 12:23:09 PM[2017-03-21T11:21:30,930][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/sda2)]], net usable_space [374.4gb], net total_space [457.2gb], spins? [possibly], types [ext4]
3/21/2017 12:23:09 PM[2017-03-21T11:21:30,931][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] heap size [1.9gb], compressed ordinary object pointers [true]
3/21/2017 12:23:09 PM[2017-03-21T11:21:31,197][INFO ][o.e.n.Node               ] node name [ZVc9ZHX] derived from node ID [ZVc9ZHX3RROOSsuo2ebnaw]; set [node.name] to override
3/21/2017 12:23:09 PM[2017-03-21T11:21:31,204][INFO ][o.e.n.Node               ] version[5.2.0], pid[46], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/3.13.0-93-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_121/25.121-b13]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,287][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [aggs-matrix-stats]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,288][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [ingest-common]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,288][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-expression]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,290][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-groovy]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,290][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-mustache]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,291][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-painless]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,291][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [percolator]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,291][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [reindex]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,291][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [transport-netty3]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,292][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [transport-netty4]
3/21/2017 12:23:09 PM[2017-03-21T11:21:47,293][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded plugin [x-pack]
3/21/2017 12:23:09 PM[2017-03-21T11:22:48,775][INFO ][o.e.n.Node               ] [] initializing ...
3/21/2017 12:23:09 PM[2017-03-21T11:22:49,294][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/sda2)]], net usable_space [374.4gb], net total_space [457.2gb], spins? [possibly], types [ext4]
3/21/2017 12:23:09 PM[2017-03-21T11:22:49,295][INFO ][o.e.e.NodeEnvironment    ] [ZVc9ZHX] heap size [1.9gb], compressed ordinary object pointers [true]
3/21/2017 12:23:09 PM[2017-03-21T11:22:49,457][INFO ][o.e.n.Node               ] node name [ZVc9ZHX] derived from node ID [ZVc9ZHX3RROOSsuo2ebnaw]; set [node.name] to override
3/21/2017 12:23:09 PM[2017-03-21T11:22:49,467][INFO ][o.e.n.Node               ] version[5.2.0], pid[46], build[24e05b9/2017-01-24T19:52:35.800Z], OS[Linux/3.13.0-93-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_121/25.121-b13]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,558][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [aggs-matrix-stats]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,559][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [ingest-common]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,559][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-expression]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,568][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-groovy]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,568][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-mustache]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,578][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [lang-painless]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,579][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [percolator]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,579][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [reindex]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,579][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [transport-netty3]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,579][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded module [transport-netty4]
3/21/2017 12:23:09 PM[2017-03-21T11:23:04,583][INFO ][o.e.p.PluginsService     ] [ZVc9ZHX] loaded plugin [x-pack]

any Idea why elasticsearch is not running ?

latest, 600: ELKX 6.0.0. doesn't start

docker run:

docker run --env ELASTIC_BOOTSTRAP_PASSWORD=woow -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name elkx3 sebp/elkx

output:

`ERROR: Setting [bootstrap.pass] does not exist in the keystore.

  • Starting periodic command scheduler cron [ OK ]
  • Starting Elasticsearch Server [ OK ]
    waiting for Elasticsearch to be up (1/30)
    waiting for Elasticsearch to be up (2/30)
    waiting for Elasticsearch to be up (3/30)
    waiting for Elasticsearch to be up (4/30)
    waiting for Elasticsearch to be up (5/30)
    waiting for Elasticsearch to be up (6/30)
    waiting for Elasticsearch to be up (7/30)
    waiting for Elasticsearch to be up (8/30)
    waiting for Elasticsearch to be up (9/30)
    waiting for Elasticsearch to be up (10/30)
    waiting for Elasticsearch to be up (11/30)
    waiting for Elasticsearch to be up (12/30)
    waiting for Elasticsearch to be up (13/30)
    waiting for Elasticsearch to be up (14/30)
    waiting for Elasticsearch to be up (15/30)
    waiting for Elasticsearch to be up (16/30)
    waiting for Elasticsearch to be up (17/30)
    waiting for Elasticsearch to be up (18/30)
    waiting for Elasticsearch to be up (19/30)
    waiting for Elasticsearch to be up (20/30)
    waiting for Elasticsearch to be up (21/30)
    waiting for Elasticsearch to be up (22/30)
    waiting for Elasticsearch to be up (23/30)
    waiting for Elasticsearch to be up (24/30)
    waiting for Elasticsearch to be up (25/30)
    waiting for Elasticsearch to be up (26/30)
    waiting for Elasticsearch to be up (27/30)
    waiting for Elasticsearch to be up (28/30)
    waiting for Elasticsearch to be up (29/30)
    waiting for Elasticsearch to be up (30/30)
    Couln't start Elasticsearch. Exiting.
    Elasticsearch log follows below.
    [2018-02-21T14:58:07,574][INFO ][o.e.n.Node ] [] initializing ...
    [2018-02-21T14:58:07,642][INFO ][o.e.e.NodeEnvironment ] [gwyVfuu] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/nvme0n1p6)]], net usable_space [11.4gb], net total_space [40gb], types [btrfs]
    [2018-02-21T14:58:07,642][INFO ][o.e.e.NodeEnvironment ] [gwyVfuu] heap size [989.8mb], compressed ordinary object pointers [true]
    [2018-02-21T14:58:07,644][INFO ][o.e.n.Node ] node name [gwyVfuu] derived from node ID [gwyVfuuXQUub1a4RWnCImw]; set [node.name] to override
    [2018-02-21T14:58:07,644][INFO ][o.e.n.Node ] version[6.0.0], pid[195], build[8f0685b/2017-11-10T18:41:22.859Z], OS[Linux/4.4.114-42-default/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12]
    [2018-02-21T14:58:07,644][INFO ][o.e.n.Node ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/elasticsearch, -Des.path.conf=/etc/elasticsearch]
    [2018-02-21T14:58:08,653][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [aggs-matrix-stats]
    [2018-02-21T14:58:08,653][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [analysis-common]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [ingest-common]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [lang-expression]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [lang-mustache]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [lang-painless]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [parent-join]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [percolator]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [reindex]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [repository-url]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [transport-netty4]
    [2018-02-21T14:58:08,654][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded module [tribe]
    [2018-02-21T14:58:08,655][INFO ][o.e.p.PluginsService ] [gwyVfuu] loaded plugin [x-pack]
    [2018-02-21T14:58:10,121][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.XPackPlugin
    [2018-02-21T14:58:10,939][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/222] [Main.cc@128] controller (64 bit): Version 6.0.0 (Build 8e6ab35cf803a2) Copyright (c) 2017 Elasticsearch BV
    [2018-02-21T14:58:11,016][INFO ][o.e.d.DiscoveryModule ] [gwyVfuu] using discovery type [zen]
    [2018-02-21T14:58:11,605][INFO ][o.e.n.Node ] initialized
    [2018-02-21T14:58:11,605][INFO ][o.e.n.Node ] [gwyVfuu] starting ...
    [2018-02-21T14:58:11,755][INFO ][o.e.t.TransportService ] [gwyVfuu] publish_address {172.17.0.2:9300}, bound_addresses {0.0.0.0:9300}
    [2018-02-21T14:58:11,773][INFO ][o.e.b.BootstrapChecks ] [gwyVfuu] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
    [2018-02-21T14:58:11,778][ERROR][o.e.b.Bootstrap ] [gwyVfuu] node validation exception
    [1] bootstrap checks failed
    [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
    [2018-02-21T14:58:11,780][INFO ][o.e.n.Node ] [gwyVfuu] stopping ...
    [2018-02-21T14:58:11,901][INFO ][o.e.n.Node ] [gwyVfuu] stopped
    [2018-02-21T14:58:11,901][INFO ][o.e.n.Node ] [gwyVfuu] closing ...
    [2018-02-21T14:58:11,908][INFO ][o.e.n.Node ] [gwyVfuu] closed
    [2018-02-21T14:58:11,910][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started
    `

Am I missing something ?

Kibana is not starting

I want to Tls communication between kibana server and browser, so i added below lines to /opt/kibana/config/kibana.yml , and kibana is not starting

server.ssl.key: /etc/ssl/private/server.key
server.ssl.certificate: /etc/ssl/certs/server.crt
xpack.security.encryptionKey: "somethingatleast32characterspasswordshouldenable"
xpack.security.sessionTimeout: 600000
xpack.reporting.encryptionKey: "9eXx*#T8'B;gzy*"

but i could able to start kibana by executing this command bin/kibana -c config/kibana.yml .... and not able start with service kibana start even error log also not generating

File name too long

Hi,

Thanks for the docker images. I've followed the guide for bootstrapping the container and running it normally. After I set the passwords for elastic and kibana to changeme and stop the container. I then restart it as follows:

docker-compose up elkx

docker-compose.yml

elkx:
  image: sebp/elkx
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"
  environment:
    - "TZ=Europe/London"
    - "ELASTICSEARCH_USER=elastic"
    - "ELASTICSEARCH_PASSWORD=changeme"
    - "KIBANA_USER=kibana"
    - "KIBANA_PASSWORD=changeme"
    - "LOGSTASH_USER=elastic"
    - "LOGSTASH_PASSWORD=changeme"

I'm getting the following error when the container starts up...

elkx_1  | touch: cannot touch '/var/log/elasticsearch/{"error":{"root_cause":[{"type":"security_exception","reason":"failedtoauthenticateuser[elastic]","header":{"WWW-Authenticate":"Basicrealm=\"security\"charset=\"UTF-8\""}}],"type":"security_exception","reason":"failedtoauthenticateuser[elastic]","header":{"WWW-Authenticate":"Basicrealm=\"security\"charset=\"UTF-8\""}},"status":401}.log': File name too long

The auth is failing but in addition it looks like elastic is trying to write JSON as a path?

I'm using sebp/elkx:622

Issue starting Logstash

Hey, I'm using version 5.6 and this is what I have for my compose file:

elkx:
image: sebp/elkx:560
ports:
- "5601:5601"
- "9200:9200"
- "5044:5044"
environment:
- ES_JAVA_OPTS="Xms512m -Xmx2g"

About as simple as it can be and I'm not making any customizations. Then I just do a docker-compose up and everything seems to start up just fine.

But when I try to create a dummy log by going this:

/opt/logstash/bin/logstash -e 'input { stdin { } } output { elasticsearch { hosts => ["localhost"] user => "elastic" password => "changeme" } }'

I get the error:

[FATAL][logstash.runner] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the "path.data" setting.

I definitely don't have multiple instances running. I don't even have any other containers running on this host. Any ideas?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.