Giter Club home page Giter Club logo

Comments (3)

Dieterbe avatar Dieterbe commented on August 15, 2024

and managing metrics assignments (using matching mechanism) is going to be one hell of manual work on 1000s of already existing metrics.

this is why consistentHashing exists. consistent assignment without any manual configuration.

TBH i never really understood why anyone would want "true" round robin. my understanding of that is that it effectively routes metrics randomly, so all endpoints would see all metric names but the values for each metric would be sprayed around randomly.

IIRC that's why i left it in the readme, to see if anyone would ask about it. Very few people have asked for it. (you and one person in #23 (comment) to which i replied it's implemented, and i don't remember why i said that. maybe i was referring to consistent hashing, because i think sometimes those terms are used interchangeably)

from carbon-relay-ng.

akamensky avatar akamensky commented on August 15, 2024

I'd rather consider true RR here as this would mean that IO of go-carbon would be spread more evenly. As I understand it (that wasn't explained in the docs) consistent hashing means that specific prefix would always be sent to the same destination, which leaves room for IO imbalance.

In our case, as we reached the SSD throughout limit, the disk IO is the bottleneck, hence the concern of actually addressing IO scaling.

It does indeed means same metric would be "sprayed" across multiple hosts. I have done some testing on very junky setup (using nginx as dumb UDP LB) and I've seen no problems with that. Use of carbonapi allows easily querying of multiple hosts containing the same metric and merging it into one response. I also seen no performance issues there, arguably performance would be even better as the read bottleneck in most cases is also IO.

from carbon-relay-ng.

Dieterbe avatar Dieterbe commented on August 15, 2024

consistent hashing means that specific prefix would always be sent to the same destination, which leaves room for IO imbalance.

it's not by prefix, but by hash of the full metricname. it generally results in a very even spread across all backends. (though there is 1 edge case where it doesn't, there is a bug about that - I think maybe that was #335 ).

If anyone opens a PR with RR support, i would review (and hopefully merge) it.

from carbon-relay-ng.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.