Comments (6)
the compactor can be configured to mount extra volumes via compactor.extraVolumes
and compactor.extraVolumeMounts
and disable the conventional storage via compactor.persistentVolume.enabled
. Does this not work for you?
Worth noting that for compactors and store-gateways doing this will not result in any data loss. Merely more work to do for the compactor and longer startup time for store-gateways (but also see #8166)
However, doing this for ingesters is very likely to result in data loss. One way to work around this is to set -blocks-storage.tsdb.flush-blocks-on-shutdown
, which only incurs temporary data loss until the compactor, store-gateway, and querier discover the flushed blocks. But an ungraceful shutdown of an ingester will lead to permanently losing data for the zone.
from mimir.
@dimitarvdimitrov I see no option to disable persistent storage fully. Options are PVC or emptyDir. Quote from chart values:
persistentVolume:
# If true compactor will create/use a Persistent Volume Claim
# If false, use emptyDir
#
enabled: true
from mimir.
you can still mount a memory-based volume and configure the compactor to use it as a working volume: set mimir.structuredConfig.compactor.data_dir
to the directory of the mount from compactor.extraVolumeMounts
. The emptyDir will be mounted by unused.
from mimir.
Can you provide an example how to mount a memory-based volume? I could not find any info about such type of k8s volumes. And I suspect that additional rights may be needed, is it really possible in rootless environment?
Nevertheless I guess some additional rows in helm templates and values would be more usable solution.
from mimir.
these are the helm values I was thinking of. Haven't ran it in a bigger cluster, but seems to work ok in k3d.
compactor:
extraVolumes:
- emptyDir:
medium: Memory
name: in-memory-data
extraVolumeMounts:
- mountPath: /volatile-data
name: in-memory-data
mimir:
structuredConfig:
compactor:
data_dir: /volatile-data
from mimir.
running ingesters with in-memory PVC is dangerous, so I don't think the chart should support this natively. Doing this for store-gateways and compactors see like less of an issue; I'm not opposed to it
from mimir.
Related Issues (20)
- mimirtool backfill - compactor: can't check block state/block validation failed HOT 1
- Support go 1.23 HOT 3
- feat: add support for docker `healthcheck` to `grafana/mimir` image HOT 3
- Adding podLabels in global section breaking parsing HOT 1
- Flaky TestIngester_PushToStorage_CircuitBreaker
- Docs: Native histogram cardinality API HOT 1
- [Helm] Add snippet to main section of nginx/gateway configuration for adding custom modules
- Can I use mimir on Kubernetes using filesystem local storage
- Compactor: writing block: closing index writer: postings offset table length/crc32
- New config option in mixin to use native histograms only in dashboards
- Add possibility do add common labels in helm chart
- Add remote write protocol 2.0 support HOT 1
- Add remote write protocol 2.0 support HOT 1
- Add support for EKS Pod Identity
- Mimir UI Prometheus 404 HOT 1
- Config is not imported Prometheus/Alertmanager HOT 1
- Docs: how to configure the alert manager HOT 2
- Removal of Liveness Probe in Loki 3.x for Query Frontend HOT 1
- /cortex.Ingester/MetricsForLabelMatchers time consumption HOT 1
- Make mimirtool not use color when pointed to non-tty output
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mimir.