How can you post counter metrics? about dogapi-rb HOT 5 CLOSED

Hey Travis,

Thanks for the question and the patch. The short answer is that there's work to be done here. Here's the situation:

• Our ruby client makes one http request per metric right now, so if you're counting anything with a non-trivial workload, it will adversely affect the performance of your app.
• We solved this in our Python client[1] by having very lightweight collection functions (gauge, counter, and so on) and batching those points when sending to the server. It also has a few options of when to send (in a thread, in process, gevent) tailored for different use cases.

So, here are the options:

• If it's a low volume thing you're instrumenting, you can use the Ruby client as is.
• If it's high volume and you need this right now, you can forward metrics to a local statsd instance and then on to datadog.
• If it's high volume and you can wait a little bit, we'll get the Ruby client up to snuff. I'm looking into timeline today.

Does that answer your question? Also, If you want to chat more, pop into our irc chat (#datadog on freenode) in an hour or so.

[1] - https://github.com/DataDog/dogapi/blob/master/src/dogapi/stats/dog_stats_api.py

from dogapi-rb.

We actually wouldn't be using Ruby for it, I was just using the ruby lib to test. I'm really looking to see if Datadog can aggregate the count data.

from dogapi-rb.

Got it. I understand. The answer is yes and no.

DataDog's API doesn't yet support a request like "add 1 to metric ABC right now". It accepts data more in the form of "the value of metric ABC right now is X". DataDog will then aggregate across hosts and tags, but basically, each host (or optionally process) needs to aggregate on the client before sending to the server. Adding true server side counters in on our roadmap, but there's some work to get there. In the meantime, the client side aggregation is pretty easily accomplished with tools like dogapi.py and statsd.

What's the source of your data? Does this sound like it'll work for you?

from dogapi-rb.

Hard to say, we run tons of servers. But the simplest scenario is:

• 100 servers
• 1 count each
• all send in data every minute let's say

I would like to see 100 in my graphs for each minute. Is that possible?

from dogapi-rb.

Yeah, this is totally possible. There are a few different ways to get your custom metrics flowing into DataDog. I'm going to e-mail you some of the options and we can chat and figure out which best fits your set-up. Cool?

One that is really clear is we need better documentation on this :)

from dogapi-rb.