shopify / doggy Goto Github PK

View Code? Open in Web Editor NEW

39.0 435.0 19.0 182 KB

License: MIT License

Ruby 99.75% Shell 0.25%

doggy's Introduction

Doggy

Doggy manages your DataDog dashboards, alerts, monitors, and screenboards.

Installation

Add this line to your Gemfile:

gem 'doggy'

And then execute:

$ bundle

Or install it yourself as:

$ gem install doggy

Authentication

To authenticate, you need to set API and APP keys for your DataDog account.

Environment variables

Export DATADOG_API_KEY and DATADOG_APP_KEY environment variables and doggy will catch them up automatically.

Optional: You can set DATADOG_BASE_HUMAN_URL environment variable if your organization uses a custom domain. This will change the urls given to users.

json (plaintext)

If you're feeling adventurous, just put plaintext secrets.json in your root object store like this:

{
  "datadog_api_key": "key1",
  "datadog_app_key": "key2"
}

Optional: Add datadog_base_human_url if your organization uses a custom domain.

Usage

# Syncs local changes to Datadog since last deploy.
$ doggy sync

# Download items. If no ID is given it will download all the items managed by dog.
$ doggy pull [IDs]

# Upload items to Datadog. If no ID is given it will push all items.
$ doggy push [IDs]

# Edit an item in WYSIWYG
$ doggy edit ID

# Delete selected items from both Datadog and local storage
$ doggy delete IDs

# Mute monitor(s) forever
$ doggy mute IDs

# Unmute monitor(s)
$ doggy unmute IDs

Multiple IDs should be separated by space.

Development

After checking out the repo, run bundle install to install dependencies.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release to create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Fork it ( https://github.com/shopify/doggy/fork )
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create a new Pull Request

doggy's People

Contributors

Stargazers

Watchers

Forkers

pallan etiennebarrie elvinefendi fujiimasashi shotat obiwancanoweme chargify bluemutedwisdom lemaral jmseaton dental-tech girasquid philcryoport evie404 aaronerickson-wf yohaneshk isabella232 qpc-github

doggy's Issues

Reliance on rugged/libgit2 rename detection can cause accidental deletions

Summary

doggy sync can mis-identify renamed files as file delete + file add pairs, resulting in the unintentional deletion of objects from Datadog.

Details

Doggy relies on Rugged::Diff#find_similar! to distinguish renamed files from additions and deletions. Since git does not directly track renames, libgit2 implements a heuristic for rename detection, converting pairs of file add and file delete deltas in the diff into rename deltas if the contents are sufficiently similar.

Doggy marks a Datadog object for deletion if a deletion delta still exists for the corresponding file after calling find_similar! to execute the rename detection algorithm.

It's possible to defeat this heuristic by making alterations to a file in the same commit as a rename. If this happens, the original add/delete deltas will remain, and Doggy will treat the object as deleted. Running doggy sync will delete the object from Datadog. Since Datadog IDs are automatically generated, this will break any previously existing links to the object, even if the object is recreated from git.

Libgit2 needs to implement rename detection in the general case, for files with arbitrary contents. We don't have the same restriction: our Datadog object files have a persistent ID, which must be preserved across renames. Instead of relying on find_similar! to distinguish deletions from renames, we should look for file add + file delete pairs for files containing the same object ID and treat these as modifications instead.

Support Groups

Currently, doggy appears to drop group definitions, which makes managed dashboards much harder to read.

Raw API

{
  "notify_list": null,
  "description": "",
  "template_variables": [],
  "is_read_only": false,
  "id": "djn-unf-9m9",
  "title": "Istio (temp)",
  "url": "/dashboard/djn-unf-9m9/istio-temp",
  "created_at": "2019-03-15T20:26:10.086473+00:00",
  "modified_at": "2019-03-15T20:34:46.287203+00:00",
  "author_handle": "[email protected]",
  "widgets": [
    {
      "definition": {
        "widgets": [
          {
            "definition": {
              "requests": [
                {
                  "q": "avg:envoy.server.live{*}",
                  "style": {
                    "line_width": "normal",
                    "palette": "dog_classic",
                    "line_type": "solid"
                  },
                  "display_type": "line"
                }
              ],
              "type": "timeseries",
              "title": "Envoy Servers"
            },
            "id": 6484880803725222
          },
          {
            "definition": {
              "requests": [
                {
                  "q": "avg:envoy.http.rq_total{*}.as_count()",
                  "style": {
                    "line_width": "normal",
                    "palette": "dog_classic",
                    "line_type": "solid"
                  },
                  "display_type": "bars"
                }
              ],
              "type": "timeseries",
              "title": "Envoy Requests"
            },
            "id": 1188283215391254
          }
        ],
        "layout_type": "ordered",
        "type": "group",
        "title": "Envoy Group"
      },
      "id": 1237966659375890
    },
    {
      "definition": {
        "requests": [
          {
            "q": "avg:istio.mesh.request.count{*}",
            "style": {
              "line_width": "normal",
              "palette": "dog_classic",
              "line_type": "solid"
            },
            "display_type": "line"
          }
        ],
        "type": "timeseries",
        "title": "Istio Requests"
      },
      "id": 3239689418669054
    }
  ],
  "layout_type": "ordered"
}

Via `doggy`

{
  "description": "",
  "graphs": [
    {
      "definition": {
        "requests": [
          {
            "q": "avg:envoy.server.live{*}",
            "style": {
              "palette": "dog_classic",
              "type": "solid",
              "width": "normal"
            },
            "type": "line"
          }
        ],
        "viz": "timeseries"
      },
      "title": "Envoy Servers"
    },
    {
      "definition": {
        "requests": [
          {
            "q": "avg:envoy.http.rq_total{*}.as_count()",
            "style": {
              "palette": "dog_classic",
              "type": "solid",
              "width": "normal"
            },
            "type": "bars"
          }
        ],
        "viz": "timeseries"
      },
      "title": "Envoy Requests"
    },
    {
      "definition": {
        "requests": [
          {
            "q": "avg:istio.mesh.request.count{*}",
            "style": {
              "palette": "dog_classic",
              "type": "solid",
              "width": "normal"
            },
            "type": "line"
          }
        ],
        "viz": "timeseries"
      },
      "title": "Istio Requests"
    }
  ],
  "id": 1113729,
  "read_only": true,
  "template_variables": [
  ],
  "title": "Istio (temp)"
}

cc: @dwradcliffe, @stefanmb

The doggy emoji

Would it be possible to make the addition of a 🐶 emoji suffix optional?

Doggy should not destroy an object when it only gets moved to another folder

Bug discovered while deploying https://github.com/Shopify/dog/pull/1127

@marc-barry

feature request: mute monitors by tag

feature request: mute monitors for a time interval instead of forever

Make doggy 2.0 compatible

Ruby 2.0 is the default Ruby version on OSX; it would be nice if doggy could work with that without having to install Ruby.

Not setup with shipit

This gem is not currently owned by [email protected], nor can we push to it using Shipit deploys.

cc @bai @andremedeiros (current owners)

Can you make [email protected] an owner please?

Add support for "doggy delete ID ID ID"

@ElvinEfendi

Related: #12.

The README states under Usage the following:

# Delete selected items from both DataDog and local storage
$ doggy delete ID ID ID

But that functionality isn't actually supported. This is issue is for adding support for this functionality.

Handle git-deletions

When running doggy push (as we do on deploy), it'd be nice if Doggy would handle object deletions in the source git repo.

Fix the bug with `doggy pull` in version 2.0.38

Here's the PR introduced it: https://github.com/Shopify/doggy/pull/42/files#diff-de7da9d4d89fab6be2343394b0004620R115

doggy pull is failing because for a given ID it's trying all three endpoints because doggy does not know if the requested resource is monitor, dashboard or a screen, so 404 for that action is expected.

Alphabetical sort of keys in written JSON

We should sort the keys, so we don't get diff-pollution when DD outputs things differently.

Pulling only one type fails

I tried doggy pull --screens, and I still got all the objects.

Add support for "doggy push ID ID ID"

The README states under Usage the following:

# Upload selected items to DataDog
$ doggy push ID ID ID

But that functionality isn't actually supported. This is issue is for adding support for this functionality.

Add support for an option to ignore untracked on doggy pull

When we do a doggy pull it pulls un untracked objects as well. It would be nice to be able to exclude these.

Required Ruby Version

Doggy requires ruby 2.1 or later because it uses required keyword arguments (https://github.com/bai/doggy/blob/a850c75ab1158aa52fdb1ce6cae8e694914d71e8/lib/doggy/cli/pull.rb#L5)

It would be convenient if the gemspec listed this using the required_ruby_version stanza.

Rugged dependecny: Needed to be updated to 0.26.0

Hi,

The rugged version is needed to be updated to "0.26.0". Currently, it is listed as "0.23.2" in the dependencies (doggy.gemspec), and there is an error while compiling:

error: dereferencing pointer to incomplete type ‘BIO {aka struct bio_st}’

The error also appears when Rugged 0.23.2 is directly installed (not as a dependency of doggy). However, there is no error when rugged 0.26.0 is directly installed.

Thanks.

Mute monitors when editing clone

Currently, one can do a doggy edit 123456 to edit an object. When the monitor is cloned it isn't muted. I think this should be the default state of a monitor currently under edit to prevent this monitor from firing. I think this can simply be done by creating the clone with the silenced option. See http://docs.datadoghq.com/api/#monitors for details.

`NoMethodError: undefined method `+' for nil:NilClass` on doggy push

What did you do?

I ran the command /home/vagrant/.gem/ruby/2.1.6/bin/doggy push 124448
What did you expect to happen?

I expected Doggy to push 124448.json to data dog
What happened instead?

Instead, what actually happened was it barfed, see stack trace below

Error details

NoMethodError: undefined method `+' for nil:NilClass
  /home/vagrant/.gem/ruby/2.1.6/gems/doggy-0.2.0/lib/doggy/model/monitor.rb:95:in `push'
  /home/vagrant/.gem/ruby/2.1.6/gems/doggy-0.2.0/lib/doggy/model/monitor.rb:16:in `block in upload'
  /home/vagrant/.gem/ruby/2.1.6/gems/doggy-0.2.0/lib/doggy/worker.rb:21:in `call'
  /home/vagrant/.gem/ruby/2.1.6/gems/doggy-0.2.0/lib/doggy/worker.rb:21:in `block (2 levels) in call'
  /home/vagrant/.gem/ruby/2.1.6/gems/thread-0.2.2/lib/thread/pool.rb:67:in `call'
  /home/vagrant/.gem/ruby/2.1.6/gems/thread-0.2.2/lib/thread/pool.rb:67:in `execute'
  /home/vagrant/.gem/ruby/2.1.6/gems/thread-0.2.2/lib/thread/pool.rb:405:in `block (2 levels) in spawn_thread'
  /home/vagrant/.gem/ruby/2.1.6/gems/thread-0.2.2/lib/thread/pool.rb:372:in `loop'
  /home/vagrant/.gem/ruby/2.1.6/gems/thread-0.2.2/lib/thread/pool.rb:372:in `block in spawn_thread'

Support subdomain based datadog URLs

Doggo supports https://app.datadoghq.com urls but if an organization has https://orgname.datadoghq.com domain name, it fails to authenticate.

Support for sorting JSON

The Datadog API returns JSON keys in different order making checking for changes with doggy pull very hard when you have a lot of dashboards, monitors and screens. Support for sorting of the JSON is needed so that when the JSON doesn't change, and only the orders does, that we don't see a change from Git's perspective.

Note that I have even seen that pulling the same object more then once results in differently ordered JSON coming from the Datadog API.

Support `doggy edit FILENAME`

Now objects can have nice filenames, it would be nice if you can refer to these filenames instead of IDs.

Can't pull screenboards

Running doggy pull 46006 didn't work. Seems like doggy doesn't pull down screenboards?

doggy pull retrieves wrong JSON on duplicate id?

Was trying to pull a screen with: doggy pull --screens 67266
should have received screen/67266 but received dash/67266 instead.

https://github.com/bai/doggy/blob/master/lib/doggy/cli/pull.rb#L33

I removed Models::Dashboard and Models::Monitor from the array and it returned the screen.

Unsure if feature or bug. FYI

Requirements are not properly defined in documentation

It looks like most of this was built for use with ShipIt, but there's nothing in the documentation. The "sync" command does not work, as it checks datadog for recent deploy events from ShipIt.

To do a pull you have to first create the "objects" directory, otherwise it fails.

Was this ever meant to be used as a standalone tool?

Add support for muting for a period of time

@ElvinEfendi

Currently, doggy mute IDs mutes monitors forever. A typical use case is to mute for a period of time. This can be done via the API: http://docs.datadoghq.com/api/#monitors

Let's add support for doggy mute IDs [time] where time is of the format 1m, 1h, 1d... This would allow us to support muting for X minutes, hours or days.

doggy push should make sure the object is read only

When an object is pushed (i.e. doggy push) it should make sure that the object is read only. This can be done via object updates on the Datadog API (http://docs.datadoghq.com/api/).

shopify / doggy Goto Github PK

doggy's Introduction

Doggy

Installation

Authentication

Environment variables

json (plaintext)

Usage

Development

Contributing

doggy's People

Contributors

Stargazers

Watchers

Forkers

doggy's Issues

Summary

Details

Raw API

Via doggy

Recommend Projects

Recommend Topics

Recommend Org

Via `doggy`