bugsnag / bugsnag-api-ruby Goto Github PK

I'm writing a script to interact with our org's bugsnag data and getting a surprising 500 fetching the related projects of an org. My code (anonymized a bit) is:

Bugsnag::Api.configure do |config|
  config.auth_token = "my-auth-token"
  config.auto_paginate = true
end

Bugsnag::Api.organization("my-org-name") # returns an object matching my expectations

Bugsnag::Api.organization("my-org-name").rels[:projects].get # errors with a  500

Exception/Stacktrace:

Bugsnag::Api::InternalServerError: GET https://api.bugsnag.com/organizations/my-org-id/projects: 500 - Error: Internal Server Error
        from /var/lib/gems/2.3.0/gems/bugsnag-api-2.0.2/lib/bugsnag/api/response/raise_error.rb:16:in `on_complete'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/response.rb:9:in `block in call'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/response.rb:61:in `on_complete'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/response.rb:8:in `call'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/rack_builder.rb:143:in `build_response'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/connection.rb:387:in `run_request'
        from /var/lib/gems/2.3.0/gems/faraday-0.14.0/lib/faraday/connection.rb:138:in `get'
        from /var/lib/gems/2.3.0/gems/sawyer-0.8.1/lib/sawyer/agent.rb:94:in `call'
        from /var/lib/gems/2.3.0/gems/sawyer-0.8.1/lib/sawyer/relation.rb:264:in `call'
        from /var/lib/gems/2.3.0/gems/sawyer-0.8.1/lib/sawyer/relation.rb:163:in `get'
        from (irb):32
        from /usr/bin/irb:11:in `<main>'

Fetching other rels (e.g. contributors) in the same manner works successfully. Fetching the same URL via curl -H "Authorization: token my-auth-token" https://api.bugsnag.com/organizations/my-org-id/projects also works successfully, so I'm working on the belief right now that something about how the gem is constructing this request disagreed with an expectation of the API.

Other details:

• Ruby 2.3.3p222

If there are other details I can provide about environment or inspection of any of the objects involved that would be helpful in understanding what caused this, please let me know. Thanks for your help.

Describe the bug

Getting organizations appears to be broken when following the readme

Steps to reproduce

1. Authenticate
2. Run org = Bugsnag::Api.organization("organization-id")
3. Scroll down to '....'

Environment

• Ruby version: ruby 2.7.7p221

Example code snippet

require 'bugsnag/api'

Bugsnag::Api.configure do |config|
  config.auth_token = "<token>"
end

org = Bugsnag::Api.organization("organization-id")

parse_query_and_convenience_headers': undefined method `fetch' for "organization-id":String (NoMethodError)

I'm hoping this isn't actually a bug but just user error on my part, but reporting it either way to clarify (also sorry if I'm reporting this in the wrong place, but it seemed like it might be an API issue instead of with the gem, but I know you guys maintain the gem as well...)

So, using the ruby gem for the Bugsnag API, if I set up a client with auto_paginate set to false in the client config, it continues to return paginated results.

For example:

2.1.5 :125 > client = Bugsnag::Api::Client.new(auth_token: [my_token], auto_paginate: false)

 => #<Bugsnag::Api::Client:0x007fca91239868 @configuration=#<Bugsnag::Api::Configuration:0x007fca91239840 @endpoint="https://api.bugsnag.com", @user_agent="Bugsnag API Ruby Gem 1.0.3", @middleware=#
<Faraday::RackBuilder:0x007fca8a50c740 @handlers=[Bugsnag::Api::Response::RaiseError, Faraday::Adapter::NetHttp], @app=#<Bugsnag::Api::Response::RaiseError:0x007fca8a952b10 @app=#<Faraday::Adapter::NetHttp:0x007fca8a952b60 @app=#<Proc:0x007fca8a952c28@/Users/aimeeault/.rvm/gems/ruby-2.1.5@warehouse/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:152 (lambda)>>>>, @auto_paginate=false, @connection_options={:headers=>{:user_agent=>"Bugsnag API Ruby Gem 1.0.3"}}, @auth_token=[removed]>> 

I then use the client to fetch errors and events counts on the last project on my account:

2.1.5 :126 > client.errors(client.projects.last.id).count
 => 30 

2.1.5 :126 > client.events(client.projects.last.id).count
 => 30 

30 being the default pagination in the API. However, if I take a look at client.projects.last, I can see that it has an errors count of 2719, much higher than 30. Overriding the paginated count in the options sent with the request seems to cap the set to 100 results (which also seems to be the client's default).

Also: To provide better context for what I'm trying to do: I just want to get daily counts of events--I excluded the start_date, end_date arguments from above just to clarify the issue but in actual use, I'm including those... so it seems wasteful to need to pull so much event data and also kind of weird that I'd have to iterate over multiple requests just to get accurate counts. I've gone through the API docs a few times but it seems like this is the only way to get that information?

Your client documentation, as well as your implementation of the auto_pagination in the Client#paginate method both rely on usage of Bugsnag::Api::Client#last_response/@last_response. Because folks will often be using the singleton instance of the client via Bugsnag::Api.client, this makes it very likely that there will be collisions when using this api in a multi-threaded environment (like Sidekiq), not exclusively, but especially if configured for auto_pagination.

I would suggest that you remove #last_response from the Client as a class attribute as well the public method, and in your auto_paginate implementation, you simply using a locally scoped variable instead.

If folks want to store the last_response in their own usage of the library, they can store it themselves in a local variable.

Describe the bug

The gem calls URI.escape in Bugsnag::Client#request:

This method is obsolete: it raises a warning in 2.7.x and has been removed in 3.x.
Escaping methods from CGI.escape should be used instead, see https://ruby-doc.org/stdlib-2.7.1/libdoc/uri/rdoc/URI/Escape.html#method-i-escape-label-Description

Be warned that CGI.escape and URI.escape produce different results, especially with the / char, eg:

URI.encode("foo/bar") # "foo/bar"
CGI.encode("foo/bar") # "foo%2Fbar"

Steps to reproduce

1. Install Ruby 3.x
2. Perform a request to any endpoint
3. See the NoMethodError being raised, ie:

NoMethodError:
  undefined method `escape' for URI:Module

Environment

• Ruby version: 3.x

Workaround

Patch the method back into URI, ie:

module URI
  def self.escape(str, unsafe = URI::UNSAFE)
    str.gsub(unsafe, &CGI.method(:escape))
  end
end

We were writing a script to pull error events from the api, to parse some associated information so that we could backfill some data that needed fixed. Upon using the ruby gem, all was going well until I realized that Bugsnag::Api.error_events does not gracefully handle being rate-limited, which ultimately came down to the paginate method.

Our solution was to override that method on Bugsnag::Api::Client, but it's something that I would expect the client library to handle, or at least call out in the docs.

module Bugsnag::Client::GracefulRateLimit
  def paginate(url, options = {}, &block)
    opts = parse_query_and_convenience_headers(options.dup)
    if configuration.auto_paginate || configuration.per_page
      opts[:query][:per_page] ||=  configuration.per_page || (configuration.auto_paginate ? 100 : nil)
    end

    data = request(:get, url, opts)

    if configuration.auto_paginate
      while @last_response.rels[:next]
        begin
          @last_response = @last_response.rels[:next].get
        rescue Bugsnag::Api::RateLimitExceeded => e
          sleep 60 and redo
        else
          if block_given?
            yield(data, @last_response)
          else
            data.concat(@last_response.data) if @last_response.data.is_a?(Array)
          end
        end
      end
    end

    data
  end
end

Bugsnag::Api::Client.prepend Bugsnag::Client::GracefulRateLimit

Hey guys,

You allow the user to register deploys through your notifier API, making reports like 'Errors introduced in the latest deploy' possible. I was wondering if you were planning on making a read API available for registered deploys? Presumably just the index/show endpoints, it would be great to have this for external reporting of bugs/deploy.

If the API already exists and is just undocumented, or if it were to be built, I would be happy to open a PR adding the functionality to this repo.

Describe the bug

While using the library to collect metadata about events/errors, due to the volume of requests easily exceeding the rate limits, I hit the rate limits often. The documentation for the API indicates that the responses will have two headers indicating the number of maximum requests per minute, and the time left to make another request within the next rate limiting window. These headers are not present on the Bugsnag::Api::RateLimitExceeded exception, making it hard to adequately handle retries.

Steps to reproduce

1. Use the bugsnag-api gem.
2. Make any request via the library enough times.
3. Receive Bugsnag::Api::RateLimitExceeded exception.
4. It only has the following methods:

[:detailed_message, :backtrace, :backtrace_locations, :set_backtrace, :cause, :full_message, :message, :exception]

None of these contain rate limiting information.

Environment

• Ruby version: 3.2.2

Example code snippet

Bugsnag::Api.error_events(project_id, error_id).each do |event|
  begin
    metadata = Bugsnag::Api.event(project_id, event.id).metaData
    pp metadata
  rescue Bugsnag::Api::RateLimitExceeded => e
    puts "Rate limit exceeded, sleeping..."
    sleep 60
    retry
  end
end

#<Bugsnag::Api::RateLimitExceeded: GET https://api.bugsnag.com/projects/xxxxevents/xxxx: 429 - >

If I configure the Bugsnag API client with a custom endpoint and auto pagination, it appears that the pagination links revert to http://localhost as the base href:

client = Bugsnag::Api::Client.new(
  auth_token: "[TOKEN]",
  endpoint: "http://bugsnag.mycompany.com",
  auto_paginate: true,
  per_page: 100,
)
client.error_events('[ERROR_ID]')
# => Bugsnag::Api::ServiceUnavailable: GET http://localhost/errors/[ERROR_ID]/events?direction=desc&offset=54f889aa3c83d80c1110fd12&per_page=100&sort=received_at: 503 - from /Users/gabriel/.rvm/gems/ruby-2.1.2/gems/bugsnag-api-1.0.2/lib/bugsnag/api/response/raise_error.rb:16:in `on_complete'

per pagination documentation, I'm attempting to pull down all the events associated with a given error. As it loops through, the 4th time through, it errors with a 400 bad request error and outputs a very large URI with several offset event_ids parameters.

error: (project, error, and event ids redacted)

Bugsnag::Api::BadRequest: GET https://api.bugsnag.com/projects/project_id/errors/error_id/events?base=2018-09-06T00%3A57%3A53Z&offset%5Bcount%5D=400&offset%5Bevent_ids%5D%5B%5D=event_id1&offset%5Bevent_ids%5D%5B%5D=event_id2&offset%5Bevent_ids%5D%5B%5D=event_id3&offset%5Bevent_ids%5D%5B%5D=event_id4&offset%5Bevent_ids%5D%5B%5D=event_id5&offset%5Bevent_ids%5D%5B%5D=event_id6&offset%5Bevent_ids%5D%5B%5D=event_id7&offset%5Bevent_ids%5D%5B%5D=event_id8&offset%5Bevent_ids%5D%5B%5D=event_id9&offset%5Bevent_ids%5D%5B%5D=event_id10&offset%5Bevent_ids%5D%5B%5D=event_id11&offset%5Bevent_ids%5D%5B%5D=event_id12&offset%5Bevent_ids%5D%5B%5D=event_id13&offset%5Bevent_ids%5D%5B%5D=event_id14&offset%5Bevent_ids%5D%5B%5D=event_id15&offset%5Bevent_ids%5D%5B%5D=event_id16&offset%5Bevent_ids%5D%5B%5D=event_id17&offset%5Bevent_ids%5D%5B%5D=event_id18&offset%5Bevent_ids%5D%5B%5D=event_id19&offset%5Bevent_ids%5D%5B%5D=event_id20&offset%5Bevent_ids%5D%5B%5D=event_id21&offset%5Bevent_ids%5D%5B%5D=event_id21&offset%5Bevent_ids%5D%5B%5D=event_id22&offset%5Bevent_ids%5D%5B%5D=event_id23&offset%5Bevent_ids%5D%5B%5D=event_id24&offset%5Bevent_ids%5D%5B%5D=event_id25&offset%5Bevent_ids%5D%5B%5D=event_id26&offset%5Bevent_ids%5D%5B%5D=event_id27&offset%5Bevent_ids%5D%5B%5D=event_id28&offset%5Bevent_ids%5D%5B%5D=event_id29&offset%5Bevent_ids%5D%5B%5D=event_id30&offset%5Bevent_ids%5D%5B%5D=event_id31&offset%5Bevent_ids%5D%5B%5D=event_id32&offset%5Bevent_ids%5D%5B%5D=event_id33&offset%5Btimestamp%5D=2018-09-05T21%3A53%3A26.000Z&offset%5Btotal_count%5D=11447&per_page=100: 400
-
from /Projects/my_app/vendor/bundle/ruby/2.3.0/gems/bugsnag-api-2.0.2/lib/bugsnag/api/response/raise_error.rb:16:in `on_complete'

code to reproduce:

client = Bugsnag::Api::Client.new(auth_token: "my-token")
project_id ="abc123"
filters = { "event.since"       => [{ type: "eq", value: "2018-09-05T12:00:00.000Z" }],
            "event.before"      => [{ type: "eq", value: "2018-09-05T23:00:00.000Z" }],
            "app.release_stage" => [{ type: "eq", value: "production" }],
            "event.severity"    => [{ type: "eq", value: "error" }, { type: "eq", value: "warning" }],
            "error.status"      => [{ type: "eq", value: "open" }],
            "event.class"       => [{ type: "eq", value: "ActiveRecord::StatementInvalid" }]
}

errors = client.errors(project_id, nil, { filters: filters })
events = client.error_events(project_id, errors[0][:id], { per_page: 100 })

last_response = client.last_response
puts Addressable::URI.unescape(last_response.rels[:next].href_template.pattern)

until last_response.rels[:next].nil?
  last_response = last_response.rels[:next].get
  puts Addressable::URI.unescape(last_response.rels[:next].href_template.pattern)
end

`

if i change per_page option to 50, it errors on the 8th time through the loop, so it seems like it gets to the same spot and blows up once it gets through 400 events. I've tried it on a few other errors, another one with a large amount of events, and a couple smaller ones and those get past the 400 mark and I eventually get a 429 RateLimit error, which is expected.

Any help would be appreciated. Thanks.

Somehow I broken hypermedia support (automatic parsing blah_url into a rels[:blah] relation object) before v1

I tried to get the request params of a few failed requests against an API endpoint of ours from Bugsnag. The request params contained datetimes. bugsnag-api seems to try to parse datetimes into Time objects. In my case, they ended up being wrong. The datetimes were formatted like this: "10:27:13 Mar 27, 2020 PDT". Without information about the formatting, it will be hard to guess the right format and parse it correctly in some cases. Datetimes probably shouldn't be parsed at all for request params.

Hey there,
I'm trying to work with you API and encountered a few issues.
I'm using the last gem version (2.0.2).

1/ Trying to use custom pagination. So I copy-pasted your documentation example:

errors = ::Bugsnag::Api.errors("XXXX", per_page: 100)
puts errors.count # 30 

The per_page does not work.

2/ Then I tried to use the auto_paginate and the per_page option:

Bugsnag::Api.configure do |config|
  config.auth_token = ENV["BUGSNAG_AUTH_TOKEN"]
  config.auto_paginate = true
  config.per_page = 100
end

And first the offset is wrong :(

D, [2018-03-29T16:54:43.424072 #24655] DEBUG -- : [httplog] GET http://api.bugsnag.com:443/projects/XXXX/errors?per_page=100 completed with status code 200 in 0.7256659999993644 seconds
D, [2018-03-29T16:54:44.188476 #24655] DEBUG -- : [httplog] GET http://api.bugsnag.com:443/projects/XXXX/errors?base=2018-03-29T14%3A54%3A43Z&offset=30&per_page=100 completed with status code 200 in 0.7253240000000005 seconds
D, [2018-03-29T16:54:45.110955 #24655] DEBUG -- : [httplog] GET http://api.bugsnag.com:443/projects/XXXX/errors?base=2018-03-29T14%3A54%3A43Z&offset=60&per_page=100 completed with status code 200 in 0.8854650000002948 seconds

Offset is by 30 instead of 100.
And secondly it doesn't handle throttling (I always end up with 429).

Also what's up with the throttling? I'm on a paid-plan and throttled after a few requests only :(

3/ I tried to use the filters option but no matter what I try it never works, here's an example:

::Bugsnag::Api.errors("XXX", filters: {"error.status": [{ "type": "eq", "value": "open" }]}).map {|e| e.status }
# D, [2018-03-29T17:04:42.157535 #25060] DEBUG -- : [httplog] GET http://api.bugsnag.com:443/projects/5326af10bbddbdd9210000cf/errors completed with status code 200 in 0.7052990000001955 seconds
# => ["open", "open", "open", "open", "open", "open", "open", "open", "open", "open", "open", "open", "open", "snoozed", "snoozed", "snoozed", "open", "open", "snoozed", "open", "snoozed", "snoozed", "snoozed", "snoozed", "open", "open", "open", "ignored", "snoozed", "open"]

Is this gem still maintained or am I doing something terribly wrong?

The trends api takes a filter parameter to allow you to narrow down the trends request, but because the options hash is merged with a {:query => {:resolution => resolution}} it's impossible to add query parameters to the request.

I have a workaround, but it involves building my own options hash and making the call like this: ::Bugsnag::Api.get("projects/#{project["id"]}/trend", options)