Should allow aggregation of pod/container metrics by deployment about kube-state-metrics HOT 8 CLOSED

kube-state-metrics intends to mirror what the Kubernetes API exposes. If there is a clear connection between objects I'm happy to expose that in a label in the kube_pod_info metric. A candidate for that could be an ownerReference, which if a Pod belongs to a ReplicaSet is set to that ReplicaSet.

Alternatively if using Prometheus, you can use relabelling rules to parse these things out of the Pods name.

More generally regarding metric and label exposure these rules apply:

• https://github.com/kubernetes/community/blob/master/contributors/devel/instrumentation.md#normalization
• #27 (comment)

from kube-state-metrics.

So we have the ReplicaSet information in the kube_pod_info metric, but as also mentioned in #27, one is usually not interested in the ReplicaSet directly, but the Deployment / DaemonSet / ... that created it. That is, the creator of the creator. Though including this two-level info could be seen as somewhat arbitrary (what if there is nothing above the ReplicaSet or a directly created pod, or what if even the Deployment was created by something higher up that you want to track?), it seems that 99% of pods are either directly rooted at something exactly 2 creation levels apart (Deployment, DaemonSet, ...) or started as standalone pods.

So I think it would be useful and not too problematic to include that information in the kube_pod_info metric somehow. The names of those objects should also not change during a pod's lifetime, meaning there shouldn't be a concern about denormalization changing all pod series here.

The question would be what to name the labels for this. We already have created_by_kind and created_by_name labels for the first parent, but what would the labels be called for the grandparent?

from kube-state-metrics.

Generally I'm all for this if we can get this information presented in a reasonable way.

The created_by_* labels are deprecated, as the underlying annotation on upstream objects is as well in favor of the OwnerReferences. The problem with that is that an object can have multiple owners, and then it's hard to create a label "owners owner", as that can be a list rather than a single value.

from kube-state-metrics.

Ah damn, wasn't aware of multiple owners. That makes the whole thing harder indeed. Do you think multiple owners will actually be common, or an exceptional thing?

from kube-state-metrics.

Already seeing that today already unfortunately, so we can't just make assumptions. While maybe not as simple as it should be, we can still solve this with a couple of recording rules in Prometheus with joins and group_left statements. I'm guessing that's what you were trying to avoid though 🙂 .

from kube-state-metrics.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

from kube-state-metrics.

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

from kube-state-metrics.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

from kube-state-metrics.