Comments (7)
Working on this, will try to come with a PR.
from thanos.
@KingRebo38 What's the query are you running in Ruler?
We are thinking about detecting the resolution rather than setting up the flag manually.
A ruler instance level flag might not work well if you have other rules which don't need 5m resolution
from thanos.
@yeya24 The Queries are something like this:
rules:
- expr: increase(kong_http_requests_total[14d])
record: kong_http_requests_total:rate:14d
- expr: increase(kong_http_requests_total[30d])
record: kong_http_requests_total:rate:30d
...
and so on
I also think the instance level flag might cause some issues, but for now I would solve it by sharding the ruler instances.
from thanos.
@KingRebo38 you can optimize these rules by reusing data from others.
rules:
- expr: increase(kong_http_requests_total[1d])
record: kong_http_requests_total:rate:1d
- expr: sum_over_time(kong_http_requests_total:rate:1d[14d]) # sum the 1d-rate over 14d
record: kong_http_requests_total:rate:14d
- expr: sum_over_time(kong_http_requests_total:rate:1d[28d]) # sum the 1d-rate over 28d
record: kong_http_requests_total:rate:28d
With this you basically have your own downsampling. As long as they are part of the same rule group, evaluation order is respected.
Edit: Ah, if your raw resolution retention is 7 days it won't work :(
from thanos.
@KingRebo38 does it work if you set max resource resolution to auto
?
from thanos.
@douglascamata If I set it to auto, it defaults to the raw data
from thanos.
You can try then setting --query-range.request-downsampled
on your Query Frontend.
from thanos.
Related Issues (20)
- External labels not applied to alerts HOT 3
- 0.35: Panic with query mode distributed HOT 1
- query: Passing `THANOS-TENANT: <tenant>` header has no effect unless `--query.enforce-tenancy` is set HOT 3
- query: different results for rate function when not dedup or using implicit step interval HOT 8
- Thanos compactor causing huge memory spikes when compacting raw blocks HOT 2
- Ruler evaluation warning false alarm caused by engine warnings HOT 1
- Expose when Rule alert labels are being overwritten by its --label(s)
- Read value of remote_user in Slow Query Logs of Query Frontend from a HTTP header HOT 3
- Thanos Receive doesn't announce external_labels which are set in hashrings.json when it works in routing and ingesting mode. HOT 1
- Issue with deduplication alogrithm in Thanos HOT 4
- Query Stats Returned with query including query bytes fetched HOT 5
- Max and min pointed at Sidecars not working on 0.35 HOT 13
- `ThanosSidecarBucketOperationsFailed` alert is flaky
- PR Title Validation
- Thanos Receive Pod is crashing with Readiness and livness Probe Failed
- Thanos ruler vs. eventual consistency of metrics
- Can Huawei's OBS storage be supported?
- Thanos React-app : Proxy server for thanos-query
- Query: update of endpoint failed...context deadline exceeded
- Thanos Chart 0.34.0 app version 12.23.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from thanos.