Comments (3)
and managing metrics assignments (using matching mechanism) is going to be one hell of manual work on 1000s of already existing metrics.
this is why consistentHashing
exists. consistent assignment without any manual configuration.
TBH i never really understood why anyone would want "true" round robin. my understanding of that is that it effectively routes metrics randomly, so all endpoints would see all metric names but the values for each metric would be sprayed around randomly.
IIRC that's why i left it in the readme, to see if anyone would ask about it. Very few people have asked for it. (you and one person in #23 (comment) to which i replied it's implemented, and i don't remember why i said that. maybe i was referring to consistent hashing, because i think sometimes those terms are used interchangeably)
from carbon-relay-ng.
I'd rather consider true RR here as this would mean that IO of go-carbon would be spread more evenly. As I understand it (that wasn't explained in the docs) consistent hashing means that specific prefix would always be sent to the same destination, which leaves room for IO imbalance.
In our case, as we reached the SSD throughout limit, the disk IO is the bottleneck, hence the concern of actually addressing IO scaling.
It does indeed means same metric would be "sprayed" across multiple hosts. I have done some testing on very junky setup (using nginx as dumb UDP LB) and I've seen no problems with that. Use of carbonapi allows easily querying of multiple hosts containing the same metric and merging it into one response. I also seen no performance issues there, arguably performance would be even better as the read bottleneck in most cases is also IO.
from carbon-relay-ng.
consistent hashing means that specific prefix would always be sent to the same destination, which leaves room for IO imbalance.
it's not by prefix, but by hash of the full metricname. it generally results in a very even spread across all backends. (though there is 1 edge case where it doesn't, there is a bug about that - I think maybe that was #335 ).
If anyone opens a PR with RR support, i would review (and hopefully merge) it.
from carbon-relay-ng.
Related Issues (20)
- Better handling when aggregationFile isn't defined HOT 1
- Suggestion: Add official support for Ubuntu Bionic and Focal HOT 7
- "too many open files" possible when using spooling? HOT 2
- kafkaMdm sending streams in strange encoding format to Kafka HOT 10
- Blocklist does not work HOT 3
- Compile fails on 32-bit ARM architecture HOT 2
- Set INSTANCE name via environment variable
- [enhancement] Support X-Script-Name header for the admin webpage
- Release Binaries are deleted on CircleCI HOT 1
- Spoolfiles are not being flushed after destination comes back up
- Support parsing and forwarding of intervals lines in storage-schemas.conf files
- [feature] Kafka input
- Annotations support for graphite
- can you help me to compile carbon-relay for Linux A8 architecture
- While compiling carbon-relay for A8 (ARM ) Facing errors in flag_parsers.go
- service_is internal metrics
- Version 1.4.4 did not successfully release HOT 1
- FR: Custom HTTP base path
- pickle.go: Unrecognized type *big.Int for value
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from carbon-relay-ng.