I tried these steps on minikube :

minikube start --memory 4096
kubectl apply   --filename https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml

Error from server (BadRequest): error when creating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": Role in version "v1" cannot be handled as a Role: no kind "Role" is registered for version "rbac.authorization.k8s.io/v1"
Error from server (BadRequest): error when creating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": RoleBinding in version "v1" cannot be handled as a RoleBinding: no kind "RoleBinding" is registered for version "rbac.authorization.k8s.io/v1"
Error from server (BadRequest): error when creating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1"
Error from server (BadRequest): error when creating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": ClusterRoleBinding in version "v1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1"

See https://github.com/kubernetes/ingress/tree/master/examples/external-auth/nginx

https://github.com/giantswarm/docker-filebeat

We need to write docs for this recipe. See https://github.com/giantswarm/kubernetes-prometheus/tree/master/docs for some inspiration

Like over there: https://github.com/giantswarm/kubernetes-prometheus/tree/master/manifests

The sudo sysctl -w vm.max_map_count=262144 command is app specific so we should try to set it inside the pod/container.

Hector suggested in an internal issue (https://github.com/giantswarm/giantswarm/issues/854#issuecomment-254480089) to run a pribileged init container for ES that makes those modifications in the host?

Need to think about how to include it best so it persists over restarts.

• Use Condition fields to allow enabling/disabling certain components.
• Write some Chart tests
• Have a look at samsung-cnct/chart-elasticsearch

Make sure these issues are addressed with that:

• #41
• #36
• #27
• #24

See https://github.com/heptio/eventrouter

Add support for structured logging via Elastic stack. This would require adding json_decode_fields support (see elastic/beats@50db534). Ideally/selfishly I would like to see a recipe for structured logging of a Spring Boot application leveraging https://github.com/logstash/logstash-logback-encoder#encoder.

I deploy the manifest-all.yaml in to kube-system namespaces and give a PersistentVolume to binding with the claim, deployment works as expected and all pods is up and running, no errors.

But if I try to configure the index pattern with fluentd-* it is unable to find.
some suggestion to figure out why?

Compare the different features of filebeat, fluentd and fluent-bit in a table.

Would be nice to have description for other Kubernetes "Laptop" clusters too.

Got below error:

sed: can't create temp file '/fluentd/etc/fluent.confXXXXXX': Read-only file system

Seems the reasone is the the below volume config is configMap:

        volumeMounts:
        - name: config
          mountPath: /fluentd/etc

And in the latest kubernetes version (1.10.0, or 1.9.*?), configMap is mounted readonly which means /fluentd/etc is readonly. However, the fluentd's entrypoint.sh would write the config that causes the error.

See this issue in fluentd.
fluent/fluentd-kubernetes-daemonset#90

Maybe only as an alternative. Would be good to also have the fluentd option not only because it's CNCF, but also cause people might want to ship logs to other datastores that are better supported by fluentd.

An example DS can be found here:
https://github.com/inovex/kubernetes-logging
and
http://logz.io/blog/kubernetes-log-analysis/

Tasks:

• Create Fluentd Docker image
• Create Kubernetes manifests
• Document configuration settings
• Document special config for Minikube

From there: http://label-schema.org/rc1/

I am using minikube 0.22.3 which does not use boot2docker.

The Yaml validation fails.
error: error validating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": error validating data: the server could not find the requested resource; if you choose to ignore these errors, turn validation off with --validate=false

Error from server (BadRequest): error when creating "https://raw.githubusercontent.com/giantswarm/kubernetes-elastic-stack/master/manifests-all.yaml": the API version in the data (batch/v2alpha1) does not match the expected API version (batch/v1beta1)

I'm running Kubernetes 1.8.0

Currently, manifests-all.yaml requires the parameter --namespace=default. It is desirable though to be able to deploy into some other namespace, e.g. kube-system, to keep default less populated.

Hello,

Thank you for the script! It seems that as of right now the deployed kibana is open for anyone to see, how would you go about securing the access ?

Thanks,
Max

It looks like there is extra text in the filebeat DamemonSet section of the manifests-all.yaml which causes the install to fail.

Filebeat and Ingest might not be enough to replace Logstash yet. Let's check on this.

Maybe see:

• https://github.com/pires/kubernetes-elasticsearch-cluster

This looks interesting:

• Kubernetes discovery plugin for Elasticsearch