Comments (5)
Hey Travis,
Thanks for the question and the patch. The short answer is that there's work to be done here. Here's the situation:
- Our ruby client makes one http request per metric right now, so if you're counting anything with a non-trivial workload, it will adversely affect the performance of your app.
- We solved this in our Python client[1] by having very lightweight collection functions (gauge, counter, and so on) and batching those points when sending to the server. It also has a few options of when to send (in a thread, in process, gevent) tailored for different use cases.
So, here are the options:
- If it's a low volume thing you're instrumenting, you can use the Ruby client as is.
- If it's high volume and you need this right now, you can forward metrics to a local statsd instance and then on to datadog.
- If it's high volume and you can wait a little bit, we'll get the Ruby client up to snuff. I'm looking into timeline today.
Does that answer your question? Also, If you want to chat more, pop into our irc chat (#datadog on freenode) in an hour or so.
[1] - https://github.com/DataDog/dogapi/blob/master/src/dogapi/stats/dog_stats_api.py
from dogapi-rb.
We actually wouldn't be using Ruby for it, I was just using the ruby lib to test. I'm really looking to see if Datadog can aggregate the count data.
from dogapi-rb.
Got it. I understand. The answer is yes and no.
DataDog's API doesn't yet support a request like "add 1 to metric ABC right now". It accepts data more in the form of "the value of metric ABC right now is X". DataDog will then aggregate across hosts and tags, but basically, each host (or optionally process) needs to aggregate on the client before sending to the server. Adding true server side counters in on our roadmap, but there's some work to get there. In the meantime, the client side aggregation is pretty easily accomplished with tools like dogapi.py and statsd.
What's the source of your data? Does this sound like it'll work for you?
from dogapi-rb.
Hard to say, we run tons of servers. But the simplest scenario is:
- 100 servers
- 1 count each
- all send in data every minute let's say
I would like to see 100 in my graphs for each minute. Is that possible?
from dogapi-rb.
Yeah, this is totally possible. There are a few different ways to get your custom metrics flowing into DataDog. I'm going to e-mail you some of the options and we can chat and figure out which best fits your set-up. Cool?
One that is really clear is we need better documentation on this :)
from dogapi-rb.
Related Issues (20)
- Error when deploy with capistrano is finished HOT 5
- Loading capistrano before setting application name breaks HOT 2
- v1.29.0 is tagged inconsistently with the rest of the project HOT 1
- Capistrano Integration fails with NoMethodError: undefined method `tty?' for #<Capistrano::Datadog::LogCapture:0x007fce14253070> HOT 1
- Metric type support is out of date HOT 3
- emit_point appends unspecified tags HOT 3
- Missing release v1.33.0 HOT 1
- Gem 1.37.0 failing on chef-client 11 (ruby 1.9) HOT 2
- Emitting event using Dogapi in Chef 12 returns with an empty event HOT 2
- Docs should call out how to add tags when emitting a metric
- Please release new version of gem HOT 1
- Capistrano integration does not allow to specify api host HOT 3
- Response codes are handled as integers, but are strings HOT 3
- Add support for self-signed certificates
- Add method to list Service Level Objectives HOT 2
- Circular loading warnings
- Is there a way to send events in a background thread? HOT 1
- #search_service_level_objective appears to not work as expected HOT 1
- Ruby 3.0 support
- Deprecated?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dogapi-rb.