komljen / helm-charts Goto Github PK

View Code? Open in Web Editor NEW

154.0 154.0 102.0 2.06 MB

Helm charts - Apps for Kubernetes

Makefile 1.05% Smarty 98.95%

charts helm kubernetes

helm-charts's People

Contributors

Stargazers

Watchers

Forkers

njdaniel rmccomb-bnet frankmariette cbluth rajr0 liche60 abhilash07 zerodeth dramer-817 icto n00bc0der89 peidong-hu jseparovic pathcl rajeev214 bainss jonniespratley javajon xmarlem andrespineros jadsonreis tuanpembual vdamianchicon strangerxxx sidharthanup bayazee manukasa cybersuv bhaskar-chenchu fvbock caruccio advishnuprasad xinsun-leo bernardoaguilar hitomidennis heronrs daringanitch mansilq victorhdamian arashkaffamanesh raj11646 goglides marbaj imransysg hutchison-technologies glnsagar mkjonker openshift-training binnyrs tb2186 trinnovative shrwan neowisard pwilken sif7en sblogx hykuan qeek-dev opensource-thg gen1321 dianplus quintype hbceylan bubble501 dominik-k steve-perkins ewinds proddam tarjei400 kangtiann aland-zhang aransh horcu rajattyagiin harness-training-dept arunksingh16 nigelcas babatunde-abdulquddus rohitcricket momerkic sssd-dev mendrugory bezilla jvanmalder vijaynetha remotestate witold-gren clix-dev-llc devopstoday11 marvel-works jithin89 mattfellows doytsujin rahulkumarsahu richasonys kathiravanmct127 dhipu009 shankarskill adrianohf snr1988

helm-charts's Issues

Kibana throwing error

Error states
Unable to connect to Elasticsearch at http://elasticsearch-efk-cluster:9200.

I see that is defined in the yaml file. How do I get around this?
**hoping I get a quick response **

Please add a permissive license

I would like to utilize your work in one of my projects. Would it be possible for you to add a permissive license like apache or MIT to your project?

Thanks!

gcp pvc never binds

Thanks for the elasticsearch operator and cluster
Pod has unbound PersistentVolumeClaims never resolves. Any suggestions?

here is my manifest

elasticsearch:
image:
repository: quay.io/pires/docker-elasticsearch-kubernetes
tag: "6.4.2"
name: efk-cluster
clientReplicas: 2
masterReplicas: 3
dataReplicas: 3
dataVolumeSize: 10Gi
javaOpts: "-Xms1024m -Xmx1024m"
zones:
- us-central1-f
- us-central1-b
- us-central1-a
resources:
requests:
memory: 1024Mi
cpu: 200m
limits:
memory: 2048Mi
cpu: '1'
storage:
storage-class-provisioner: kubernetes.io/gce-pd

Elasticsearch pod fails with `panic: runtime error: slice bounds out of range`

Hi,

I was using your helm repo for a PoC on EFK. When I try to deploy it on a local k8s cluster i.e, on docker-for-desktop on MacOS, I get an error on the elasticsearch operator pod. Any idea regarding this error? Help is much appreciated.

The log reads as below before terminating.

time="2019-02-27T09:08:23Z" level=info msg="elasticsearch operator starting up!"
time="2019-02-27T09:08:23Z" level=info msg="Using Variables:"
time="2019-02-27T09:08:23Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2019-02-27T09:08:23Z" level=info msg="Using InCluster k8s config"
time="2019-02-27T09:08:23Z" level=info msg="SKIPPING: already exists \"elasticsearchclusters.enterprises.upmc.com\"\n"
time="2019-02-27T09:08:24Z" level=info msg="Daemonset &DaemonSet{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:elasticsearch-operator-sysctl,GenerateName:,Namespace:default,SelfLink:/apis/extensions/v1beta1/namespaces/default/daemonsets/elasticsearch-operator-sysctl,UID:2d14c55c-3a6c-11e9-baac-025000000001,ResourceVersion:55607,Generation:1,CreationTimestamp:2019-02-27 08:46:29 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{k8s-app: elasticsearch-operator,},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:DaemonSetSpec{Selector:&k8s_io_apimachinery_pkg_apis_meta_v1.LabelSelector{MatchLabels:map[string]string{k8s-app: elasticsearch-operator,},MatchExpressions:[],},Template:k8s_io_api_core_v1.PodTemplateSpec{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{k8s-app: elasticsearch-operator,},Annotations:map[string]string{},OwnerReferences:[],Finalizers:[],ClusterName:,Initializers:nil,},Spec:PodSpec{Volumes:[],Containers:[{sysctl-conf busybox:1.26.2 [sh -c sysctl -w vm.max_map_count=262166 & while true; do sleep 86400; done] []  [] [] [] {map[cpu:{{10 -3} {<nil>} 10m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}] map[cpu:{{10 -3} {<nil>} 10m DecimalSI} memory:{{52428800 0} {<nil>} 50Mi BinarySI}]} [] nil nil nil /dev/termination-log File IfNotPresent &SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,} false false false}],RestartPolicy:Always,TerminationGracePeriodSeconds:*30,ActiveDeadlineSeconds:nil,DNSPolicy:ClusterFirst,NodeSelector:map[string]string{},ServiceAccountName:,DeprecatedServiceAccount:,NodeName:,HostNetwork:false,HostPID:true,HostIPC:false,SecurityContext:&PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,},ImagePullSecrets:[],Hostname:,Subdomain:,Affinity:nil,SchedulerName:default-scheduler,InitContainers:[],AutomountServiceAccountToken:nil,Tolerations:[],HostAliases:[],PriorityClassName:,Priority:nil,},},UpdateStrategy:DaemonSetUpdateStrategy{Type:OnDelete,RollingUpdate:nil,},MinReadySeconds:0,TemplateGeneration:1,RevisionHistoryLimit:*10,},Status:DaemonSetStatus{CurrentNumberScheduled:1,NumberMisscheduled:0,DesiredNumberScheduled:1,NumberReady:1,ObservedGeneration:1,UpdatedNumberScheduled:1,NumberAvailable:1,NumberUnavailable:0,CollisionCount:nil,},} already exist, skipping creation ..."
time="2019-02-27T09:08:24Z" level=info msg="Found cluster: efk-cluster"
time="2019-02-27T09:08:24Z" level=info msg="use-ssl false"
time="2019-02-27T09:08:24Z" level=info msg="Found 1 existing clusters "
time="2019-02-27T09:08:24Z" level=info msg="Watching for elasticsearch events..."
time="2019-02-27T09:08:24Z" level=info msg="Process Elasticsearch Event ADDED"
time="2019-02-27T09:08:24Z" level=info msg="--------> Received ElasticSearch Event!"
time="2019-02-27T09:08:24Z" level=info msg="-----> Stop scheduler efk-cluster-logging"
time="2019-02-27T09:08:24Z" level=info msg="Found cluster: efk-cluster"
time="2019-02-27T09:08:24Z" level=info msg="use-ssl false"
time="2019-02-27T09:08:24Z" level=info msg="Using [quay.io/pires/docker-elasticsearch-kubernetes:6.4.2] as image for es cluster"
time="2019-02-27T09:08:24Z" level=info msg="use-ssl false"
time="2019-02-27T09:08:25Z" level=info msg="--------> ElasticSearch Event finished!"
panic: runtime error: slice bounds out of range

goroutine 28 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc42000ae20, 0xc420736c60, 0x0, 0x0)
	/Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:248 +0x268
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc4200a6180, 0xc42000ae20, 0xc4202853e0, 0xc4200c66c0, 0xc420044650)
	/Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:107 +0x1f8
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:103 +0x78

StorageClass issue

According to comments in this post : https://akomljen.com/get-kubernetes-logs-with-efk-stack-in-5-minutes/, it seems a lot of people struggle to get it running.

k get pods -n logging                                      
NAME                                                 READY   STATUS    RESTARTS   AGE
efk-kibana-fc4bbccb7-ppc9x                           0/1     Running   0          14m
es-client-efk-cluster-857fd4c567-ngfwn               0/1     Running   3          13m
es-data-efk-cluster-default-0                        0/1     Pending   0          13m
es-master-efk-cluster-default-0                      0/1     Running   0          13m
es-operator-elasticsearch-operator-fbbd9556c-wv8jn   1/1     Running   1          19m
fluent-bit-b65vt                                     1/1     Running   0          14m
fluent-bit-rx2gk                                     1/1     Running   0          14m

I'm currently in such a state as well. I installed the chart through terraform helm provider. Installation went fine, but it doesn't run as the volumes couldn't be created. Would it be possible to expose a storage class or an option in the values file ? And document that so it would work with other IaaS than aws ?

Kibana is unable to connect to elasticsearch

Error states
Unable to connect to Elasticsearch at http://elasticsearch-efk-cluster:9200.

I see that is defined in the yaml file. How do I get around this?
**hoping I get a quick response **

Problem with K8s 1.17

"helm install --name efk --namespace logging akomljen-charts/efk" not working on k8s 1.17.

Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "DaemonSet" in version "extensions/v1beta1", unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta1"]

fluent-bit could not pack/validate JSON response

fluent-bit got wrong message as follow:

fluent-bit [2019/02/21 13:31:46] [error] [out_es] could not pack/validate JSON response
fluent-bit-wcrhs fluent-bit {"took":1866,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"3nxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":104254,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"33xBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103965,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"4HxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103966,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"4XxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103885,"_primary_term":1
fluent-bit-wcrhs fluent-bit [2019/02/21 13:31:46] [ warn] [out_es] Elasticsearch error
fluent-bit-wcrhs fluent-bit {"took":1866,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"3nxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":104254,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"33xBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103965,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"4HxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103966,"_primary_term":1,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.20","_type":"flb_type","_id":"4XxBEGkBGbBzgNoBPtAW","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":103885,"_primary_term":1,"status":201}},{"
fluent-bit-vt5xb fluent-bit [2019/02/21 13:31:46] [error] [out_es] could not pack/validate JSON response

Provide option to configure the node selector for elastic search

Is there an option to mention the node selector in the efk stack helm chart for elasticsearch nodes?

fluent-bit logs errors

The fluent-bit pod ( on minikube ) shows a lot of errors:

{"took":25,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.05.16","_type":"flb_type","_id":"5Mf2wWoBjkwQFkQkrbET","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Duplicate field '@timestamp'\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@5394d44b; line: 1, column: 816]"}}}},{"index":{"_index":"kubernetes_cluster-2019.05.16","_type":"flb_type","_id":"5cf2wWoBjkwQFkQkrbET","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"i_o_exception","reason":"Duplicate field '@timestamp'\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@6c337d70; line: 1, column: 288]"}}}},{"index":{"_index":"kubernetes_cluster-2019.05.16","_type":"flb_type","_id":"5sf2wWoBjkwQFkQkrbET","status":400,"error":{"type":"mapper_parsing_ex

fluent/fluent-bit#628 references this also.

Not sure what version you use, is it 1.0.4? :

$ kc describe pod/fluent-bit-44gtk -n logging| grep Image
    Image:          fluent/fluent-bit:1.0.4

In your chart I see version 1.3.0 ( confusing for me , sorry )

What is the correct way to solve this ?

Submit Descheduler to official chart repository

Would it be possible for you to submit your descheduler chart to the official repository?

Is it not possible to upgrade to latest version of Kibana?

tried upgrading to later versions of kibana, but that seems to be non compatible with ES versions and we are not using official version of ES.

How to go forward about it ? 🤔

EFK Chart must provide support for authentication and internal security with SSL

EFK Chart must provide support for authentication and internal security with SSL

Persistent Volumes

Seems like your EFK chart is not tied to PersistentVolumes. Am I right? Any plans to add this?

Lacking critical tag in CronJob pod template

The pod that the cronjob creates for the eviction has no annotations.

jobTemplate:
    metadata:
      labels:
        app: {{ template "descheduler.name" . }}
        release: {{ .Release.Name }}
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: "true"
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
    spec:
      template:
        metadata:
          annotations:
            scheduler.alpha.kubernetes.io/critical-pod: "true"
            checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
...
...
...

Error

Hi,

I have getting an error while installing efk stack
D:\MACHINE LEARNING\Kubernates Microservices>helm --kubeconfig "D:\MACHINE LEARNING\Kubernates Microservices\hcl-kubeConfig" install --name efk --namespace logging akomljen-charts/efk
2018/11/24 20:43:20 warning: skipped value for env: Not a table.
2018/11/24 20:43:20 warning: skipped value for env: Not a table.
2018/11/24 20:43:20 warning: skipped value for env: Not a table.
2018/11/24 20:43:20 warning: skipped value for env: Not a table.
Error: validation failed: unable to recognize "": no matches for kind "ElasticsearchCluster" in version "enterprises.upmc.com/v1"

Will you able to please help ?

[EFK] Erratic pod start

I have installed and uninstalled the chart several times, but there is no way for the pods to start correctly. They seem to be crashing one after another mostly for not being able to resolve elasticsearch-discovery-efk-cluster. This happens even when the master node is apparently running correctly.

Any clues?

NAME                                                  READY     STATUS             RESTARTS   AGE
efk-kibana-76b967c8db-ll6jm                           0/1       CrashLoopBackOff   9          32m
es-client-efk-cluster-645d47fccc-wg4xp                1/1       Running            7          32m
es-data-efk-cluster-default-0                         1/1       Running            8          32m
es-master-efk-cluster-default-0                       0/1       CrashLoopBackOff   9          32m
es-operator-elasticsearch-operator-5977ff7c98-rwqng   1/1       Running            0          1d
fluent-bit-q4z5v                                      1/1       Running            0          32m
fluent-bit-qsql6                                      1/1       Running            0          32m
fluent-bit-vrqvx                                      1/1       Running            0          32m

Thanks!

`.Capabilities.APIVersions.` returns `rbac.authorization.k8s.io/v1alpha1` with jx step helm install instead of `rbac.authorization.k8s.io/v1`

When I try to install this chart with jx step helm install, the apiVersion is not supported and the installation of the clusterrole and clusterrolebinding fails. Is there a way to change this behaviour?

Add apm-server support

Are there any plans to add a chart for apm-server?

es-operator pod keeps Crashlooping

The operator container from the efk helm chart keeps crashing with the following output:

time="2019-11-18T15:51:40Z" level=info msg="elasticsearch operator starting up!"
time="2019-11-18T15:51:40Z" level=info msg="Using Variables:"
time="2019-11-18T15:51:40Z" level=info msg="   enableInitDaemonset: true"
time="2019-11-18T15:51:40Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2019-11-18T15:51:40Z" level=info msg="Using InCluster k8s config"
time="2019-11-18T15:51:41Z" level=info msg="SKIPPING: already exists \"elasticsearchclusters.enterprises.upmc.com\""
time="2019-11-18T15:51:43Z" level=info msg="Daemonset logging/elasticsearch-operator-sysctl already exist, skipping creation ..."
time="2019-11-18T15:51:43Z" level=info msg="Found cluster: efk-cluster"
time="2019-11-18T15:51:43Z" level=info msg="use-ssl false"
time="2019-11-18T15:51:43Z" level=info msg="Found 1 existing clusters "
time="2019-11-18T15:51:43Z" level=info msg="Watching for elasticsearch events..."
time="2019-11-18T15:51:44Z" level=info msg="Process Elasticsearch Event ADDED"
time="2019-11-18T15:51:44Z" level=info msg="--------> Received ElasticSearch Event!"
time="2019-11-18T15:51:44Z" level=info msg="-----> Stop scheduler efk-cluster-logging"
time="2019-11-18T15:51:44Z" level=info msg="Found cluster: efk-cluster"
time="2019-11-18T15:51:44Z" level=info msg="use-ssl false"
time="2019-11-18T15:51:44Z" level=info msg="Using [quay.io/pires/docker-elasticsearch-kubernetes:6.4.2] as image for es cluster"
time="2019-11-18T15:51:44Z" level=info msg="use-ssl false"
time="2019-11-18T15:51:46Z" level=info msg="--------> ElasticSearch Event finished!"
panic: runtime error: slice bounds out of range

goroutine 31 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processMasterPodEvent(0xc000380120, 0xc000314000, 0x10b0148, 0x4)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:294 +0x612
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).processPodEvent(0xc000380120, 0xc000314000, 0x0, 0x0)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:277 +0xfb
github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents.func1(0xc000390180, 0xc000380120, 0xc00030f7a0, 0xc000390000, 0xc00036a180)
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:109 +0x1ed
created by github.com/upmc-enterprises/elasticsearch-operator/pkg/processor.(*Processor).WatchDataPodEvents
	/home/steve/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/processor/processor.go:105 +0x78

Is this expected behaviour? The rest of the EFK application runs nicely

Error: This command needs 1 argument: chart name

[root@xxx ~]# helm search efk
NAME CHART VERSION APP VERSION DESCRIPTION
akomljen-charts/efk 0.1.0 1.0 A Helm chart for EFK stack

[root@xxx efk]# helm install akomljen-charts/efk -name test --namespace=efk
Error: This command needs 1 argument: chart name

When I use helm to install efk, it prompts error of chart name. How should I install it correctly? Thanks for your time and guide!

Error: apiVersion "enterprises.upmc.com/v1" not available.

Getting following error while trying to do helm install.

$ helm install --name efk --namespace logging akomljen-charts/efk
Error: could not find tiller

$ helm init
$HELM_HOME has been configured at /Users/viaggarw/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!

$ helm install --name efk --namespace logging akomljen-charts/efk
2018/09/01 10:51:21 warning: skipped value for env: Not a table.
2018/09/01 10:51:21 warning: skipped value for env: Not a table.
2018/09/01 10:51:21 warning: skipped value for env: Not a table.
2018/09/01 10:51:21 warning: skipped value for env: Not a table.
Error: apiVersion "enterprises.upmc.com/v1" in efk/charts/elasticsearch/templates/cluster.yaml is not available

The Elastic nodes are not able to start.

The heathcheck and readiness probe is always failing for master and data node. Do they need more time to start?

es-data-efk-cluster-default-0 0/1 CrashLoopBackOff 5 14m
es-master-efk-cluster-default-0 0/1 CrashLoopBackOff 6 14m

Events from the pods.


  Warning  Unhealthy               7m12s                  kubelet, ip-172-20-35-37.ec2.internal  Liveness probe failed: Get http://100.96.3.157:9200/_nodes/_local: dial tcp 100.96.3.157:9200: connect: connection refused
  Normal   Pulled                  4m39s (x4 over 10m)    kubelet, ip-172-20-35-37.ec2.internal  Container image "quay.io/pires/docker-elasticsearch-kubernetes:6.4.2" already present on machine
  Normal   Created                 4m39s (x4 over 10m)    kubelet, ip-172-20-35-37.ec2.internal  Created container
  Normal   Started                 4m39s (x4 over 10m)    kubelet, ip-172-20-35-37.ec2.internal  Started container
  Warning  Unhealthy               4m19s (x6 over 7m12s)  kubelet, ip-172-20-35-37.ec2.internal  Readiness probe failed: dial tcp 100.96.3.157:9300: connect: connection refused

Nothing in the pod logs, just this.
➜ ~ kubectl logs es-master-efk-cluster-default-0
Starting Elasticsearch 6.4.2
Changing ownership of /data folder

Regards,
Ashish

Trying to get efk stack working with efs

Using the efs-provisioner and is working well with other apps. Using the following configs it still creates ebs/gp2 volumes.

 storage:
   # e.g. gp2 for AWS
   type: "efs"
   # e.g. kubernetes.io/aws-ebs for AWS
   classProvisioner: kubernetes.io/aws-efs
   # Retain or Delete
   reclaimPolicy: Retain

fluent-bit failing to get kubernetes medata

I am trying to follow your post
Get Kubernetes Logs with EFK Stack in 5 Minutes using minikube 0.28.1/kubernetes 1.10.0 and I was/am running in a few issues:

First thing was that

helm install --name es-operator \
    --namespace logging \
    --set rbac.create=false \
    akomljen-charts/elasticsearch-operator

failed to set up the CRD. Container log was showing:

time="2018-08-03T05:03:07Z" level=info msg="elasticsearch operator starting up!"
time="2018-08-03T05:03:07Z" level=info msg="Using Variables:"
time="2018-08-03T05:03:07Z" level=info msg="   baseImage: upmcenterprises/docker-elasticsearch-kubernetes:6.1.3_0"
time="2018-08-03T05:03:07Z" level=info msg="Using InCluster k8s config"
panic: customresourcedefinitions.apiextensions.k8s.io "elasticsearchclusters.enterprises.upmc.com" is forbidden: User "system:serviceaccount:logging:es-operator-elasticsearch-operator" cannot get customresourcedefinitions.apiextensions.k8s.io at the cluster scope

goroutine 1 [running]:
github.com/upmc-enterprises/elasticsearch-operator/pkg/k8sutil.(*K8sutil).CreateKubernetesCustomResourceDefinition(0xc420191620, 0x1045e88, 0x6)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/k8sutil/k8sutil.go:218 +0x64a
github.com/upmc-enterprises/elasticsearch-operator/pkg/controller.(*Controller).init(0xc420171e68, 0x1046fa7, 0xc420171db0)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/controller/controller.go:69 +0x32
github.com/upmc-enterprises/elasticsearch-operator/pkg/controller.(*Controller).Run(0xc420171e68, 0x1045e88, 0x6)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/pkg/controller/controller.go:58 +0x2f
main.Main(0xc4200ac058)
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/cmd/operator/main.go:111 +0x58e
main.main()
        /Users/stevesloka/godev/src/github.com/upmc-enterprises/elasticsearch-operator/cmd/operator/main.go:144 +0x22

which looks like an auth issue to me.

Downloaded 0.2 at https://raw.githubusercontent.com/upmc-enterprises/elasticsearch-operator/master/example/controller.yaml, changed the namespace to logging and deployed it. That created the CRD and

helm install --name efk \
    --namespace logging \
    akomljen-charts/efk

installed the operator.

However, fluent-bit fails to get POD metadata from the apiserver:

[2018/08/03 05:16:11] [ info] [engine] started (pid=1)
[2018/08/03 05:16:11] [ info] [filter_kube] https=1 host=kubernetes.default.svc port=443
[2018/08/03 05:16:11] [ info] [filter_kube] local POD info OK
[2018/08/03 05:16:11] [ info] [filter_kube] testing connectivity with API server...
[2018/08/03 05:16:11] [ warn] [filter_kube] could not get meta for POD fluent-bit-2vvwm

Turned out to be an auth issue and --set fluent-bit.rbac.create=true did the trick.

Setting the 'master-volume-size' doesn't seem to have much effect

The following configuration of akomljen-charts/efk results in both master and data volume being 512Gi in size.

  elasticsearch:
    spec:
      master-volume-size: 10Gi
      data-volume-size: 512Gi

I tried to debug but my go skills are very limited. Any clue what I'm doing wrong?

Allow custom annotations and labels on Pods

The only way to monitor what the descheduler does to the cluster is by checking the logs. For most logging tools for Kubernetes (for example Kibana), labels and annotations are critical to filter information.

Maybe something like:

...
...
...
spec:
  schedule: "{{ .Values.schedule }}"
  jobTemplate:
    metadata:
      labels:
{{ if .Values.labels }}{{ toYaml .Values.labels | indent 8 }}{{end}}
        app: {{ template "descheduler.name" . }}
        release: {{ .Release.Name }}
      annotations:
{{ if .Values.annotations }}{{ toYaml .Values.annotations | indent 8 }}{{end}}
        scheduler.alpha.kubernetes.io/critical-pod: "true"
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
    spec:
      template:
        metadata:
          labels:
{{ if .Values.labels }}{{ toYaml .Values.labels | indent 12 }}{{end}}
            app: {{ template "descheduler.name" . }}
            release: {{ .Release.Name }}
          annotations:
{{ if .Values.annotations }}{{ toYaml .Values.annotations | indent 12 }}{{end}}
            scheduler.alpha.kubernetes.io/critical-pod: "true"
            checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
        spec:
...
...
...