bugsnag / bugsnag-agent Goto Github PK

Bugsnag agent should fork off to a daemon process, detach it's STDIN/STDOUT/STDERR, and use proper logging, thus allowing it to run as a daemon.

Once this is complete, a proper SysV init script, and possibly a systemd script should be provided.

I'm on Ubuntu 12.04. The Upstart script in the README works fine except that /var/log/bugsnag.log is always empty. Running bugsnag-agent or bugsnag-agent > /var/log/bugsnag.log manually works, but if bugsnag-agent is started via the upstart script the log file never has anything in it.

Installing the agent yielded no network performance improvements for me. It turns out that PHP is not issuing a fire-and-forget and instead ends up waiting for the response from the agent but it could be something else, I am not sure.

Hopefully I am missing something obvious, any guidance would be appreciated.

The instrumentation is correct and the agent does log the incoming requests and they do show up on the bugsnag web app as well.

We were recently investigating a very high traffic bill from our datacenter to Bugsnag. It turns out that all of our servers have the following in their /var/log/upstart/bugsnag.log:

...
Cannot send request. Retrying in 5 seconds
Traceback (most recent call last):
  File "/usr/bin/bugsnag-agent", line 138, in _client
    urllib2.urlopen(req).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: Bad Request
continuing...
Enqueued 590886 bytes (119/1000)
Sending 612099 bytes (118/1000)
Cannot send request. Retrying in 5 seconds
Traceback (most recent call last):
  File "/usr/bin/bugsnag-agent", line 138, in _client
    urllib2.urlopen(req).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: Bad Request
continuing...
Enqueued 573080 bytes (119/1000)
...

It appears to have the same set of around 120 errors stuck in it's queue, and every one of the 10 threads sends one and gets the same "400 Bad Request" response, and requeues it and waits 5 seconds. That's 2 requests per second, or 172,800 per day, or at around 1MB per request, 172GB per server per day. With around 10 servers and running bugsnag-agent for around 10 days, the ~10TB bandwidth bill certainly caught us off guard.

1. Is there any way to see what problem notify.bugsnag.com has with these requests? (they come from bugsnag-php)
2. Can bugsnag-agent be made not to retry failed requests, or limit the number of retries made for each payload?

Hi,

I'm trying to set up a custom endpoint for bugsnag, by pointing my Javascript client, to a custom server, which have bugsnag client installed.

I've followed the documentation and set the config to listen to 0.0.0.0, so that I could call the endpoint from outside my server. It is up and running as expected.

I did a cUrl of the POST request and that is working as expected.

But the same is failing when I configure my Javascript client to point to the custom endpoint, in the OPTIONS call.

Expected behavior

OPTIONS method should be handled, which will be fired prior tp POST method (used by bugsnag.notify method) from Javascript Client.

Observed behavior

Error from bugsnag-agent:

[21/May/2018 06:51:22] "OPTIONS / HTTP/1.1" 501 -

Error in Browser:

Steps to reproduce

• Setup the agent as per the dcumentation.
• Configure the agent to listen to 0.0.0.0
• Configure the Javascript client to notify the errors to the new endpoint.
• Trigger an error and check error on browser and agent as well.

Version

Agent: v1.2.0
JS Client: v4

Additional information

[Insert any additional information]

Can't comment on Issues?

Some users have been unable to comment on Github issues when an adblocker extension is enabled.
We recommend temporarily disabling the extension, or if that fails, contacting [email protected].

In our case, Bugsnag does not work either with a huge amount of data (example 729k of json in the body). It also gets lost and error never reaches the Bugsnag server, or reaches it incorrectly cause it is never shown in the interface. Same test with a smaller json works just fine.
We are using bugsnag with nodejs.

We are using bugsnag-node and it has this problem as well
github.com/bugsnag/bugsnag-node

As mentioned on another issue #8 that seemed to have a similar problema on bugsnag-php
For requests over 1024 bytes, the curl HTTP client (used by PHP lib) adds Expect: 100-continue which trips up the agent and the response is extremely slow.

Adblockers is a problem for bugsnag (both sessions and actual bug reports).

This agent is great, but could use improvements. Some ideas:

• Support for forwarding sessions
• Explain/clarify security considerations, e.g. can we connect this directly to the internet (presumably!), are there any risks of abuse? What's the threat model?
• Simplify operations; package up as docker image? add deploy example?
• Maybe deploy as https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-service.html ?

If a large volume of errors are sent to bugsnag-agent for a period of time, the queue quickly hits 1000 because the sending thread can't send them to bugsnag fast enough.

Is there a way the HTTP request to notify.bugsnag.com could be done asynchronously so the sending thread doesn't have to wait for the response from bugsnag for one error before sending the next?

Alternatively, could multiple sending threads be used?