Comments (11)
In the last few days, I've given this a lot more thought.
The idea of having special pyrra.dev/team: sre
prefixed labels seems like a great idea actually.
That way commands like kubectl get slo -l pyrra.dev/team=sre
will work and we can propagate these labels to the PrometheusRule on Kubernetes too, such that kubectl get prometheusrule -l pyrra.dev/team=sre
should work as well.
It shouldn't be a big problem to extract them into the metrics either and have them as team="sre"
in the rules and alerts as well as on the UI for the list.
from pyrra.
@obitech We could certainly use this feature and would greatly appreciate the effort!
from pyrra.
I'll take a look! My yearly PTO is coming up so it might take a bit...
from pyrra.
Working on this now :)
from pyrra.
Would having the labels in the spec make it safe to propagate all labels to the alerts? This would let people use their existing alert manager configuration.
from pyrra.
Yes, that's the idea with having the labels in the spec. They would essentially become SLO labels rather than Kubernetes labels, if you will.
from pyrra.
I'm not 100% sure if we really want to add the label to the metadata.labels. It could be better to have a simliar thing to what Deployments and StatefulSets do, where we would have a spec.metadata.labels. So we can have separate labels for Kubernetes and the SLO itself.
If we want to make a separation but don't want to create a new field under the spec, we could also prefix the labels which are meant to be propagated:
apiVersion: pyrra.dev/v1alpha1
kind: ServiceLevelObjective
metadata:
name: your-slo-name-here
labels:
prometheus: k8s
role: alert-rules
pyrra.dev.alerts/team: pyrra
from pyrra.
That sounds reasonable. What would be the best way forward to implement this? I'd be happy to draft a PR for the label propagation, if you're not already on it 😌
from pyrra.
Currently, I'm not working on this yet. I'm mostly active on these: https://github.com/pyrra-dev/pyrra/projects/4
If you want to take it, go ahead! 🎉
from pyrra.
Any updates on this feature?
from pyrra.
Not sure about how other people feel about it, but now that an Alerting struct was introduced at #375 I would expect this struct to be the source of truth when it comes to alert customization.
I was thinking that this struct could customize not only alertname(#386), but also labels and annotations. Would it make sense to replace the label propagation strategy? I believe having two different ways of propagation for labels will lead to confusion.
Another strategy is to add annotation propagation with the same strategy we have for labels now: through k8s annotation propagation. So at least we have label and annotation propagation in a consistent way 🙂
from pyrra.
Related Issues (20)
- No data Budget and Availability HOT 1
- Doubt - Error Budget
- Issue when showing some SLOs (white screen) HOT 3
- Pyrra causes instability of Prometheus HOT 1
- Fix typo in Kubernetes example readme HOT 1
- Allow providing an error rate query rather than a error count query HOT 1
- ConfigMapMode not working with option --config-map-mode=true
- Webhook Panic on Unexpected Parser Expression HOT 2
- Allow Prometheus Rules to Publish Rules to Azure Managed Prometheus HOT 3
- Grafana Alerting & Mimir support HOT 9
- Latency format incorrect when using milliseconds HOT 2
- Latency SLOs - Duplicated "p100" percentile in duration graph (UI) HOT 2
- Duration graph legend shows inaccurate value when duration is greater than 1 second
- No data and generic rules not created
- How do I create a SLO for this kafka metrics
- Absent Alert not disabled when using `spec.alerting.disabled=true`
- Missing precision for Multi Burn Rate Alerts graphs (UI)
- bool_gauge SLO's budget is burning continouosly
- The GenericRules is using sum() in counter metrics HOT 3
- Add support for Prometheus offset modifier HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyrra.