exclude namespace kube-system to send logs to ElasticSearch about fluent-plugin-kubernetes_metadata_filter HOT 6 CLOSED

@viquar22 yes - add a match like this:

<match kubernetes.var.log.containers.**_kube-system_**>
  @type null
</match>

assuming you are reading container log files written by docker --log-driver=json-file

from fluent-plugin-kubernetes_metadata_filter.

Yes.
i use gitlab for deployment.
I updated my td-agent with the above config and deployed but still see the logs from "kube-system" in Kibana.

from fluent-plugin-kubernetes_metadata_filter.

@viquar22 not sure - this is a general fluentd problem, not a k8s meta plugin problem - you should ask how to debug this issue in a fluentd forum

from fluent-plugin-kubernetes_metadata_filter.

alright. Thanks for your quick response @richm

from fluent-plugin-kubernetes_metadata_filter.

@richm Hey your config works for me.
It has stopped sending logs from namespace (kube-system).
But i see logs in Kibana from same namespace (kube-system) but the pods are different.
When i try to describe (kubectl describe . . ) it doesn't exist.. i see those pod logs in the latest time stamp.

The pods i see in Kibana do not match with the ones i see in the Terminal (kubectl -n kube-system get pods). i am able to describe and login to the pods i see in the terminal and they have updated td-agent configuration.. But Kibana pods doesn't exist... how come it's possible...

I believe those Pods in Kibana are old pods that are still exist somewhere in the buffer(don't know where) and getting logs from them with latest timestamp.

Please suggest. Thanks in advance..

from fluent-plugin-kubernetes_metadata_filter.

@viquar22 I don't know - it could be many things - but I don't think this closed issue is the right place to discuss - try a kubernetes forum or a fluentd forum

from fluent-plugin-kubernetes_metadata_filter.