When the redux state is really big, the request to sentry fails due to `Request too large` about raven-for-redux HOT 8 CLOSED

Sooooo I've actually come to the conclusion that raven-for-redux and redux-raven-middleware are not the best idea. Even with all my 413 error handling and redux-undo history removal, Sentry is still aggressively trimming the "additional data" and removing large chunks of my state. See:

• https://forum.sentry.io/t/additional-data-is-getting-agressively-truncated-trimmed/4940
• getsentry/sentry-javascript#1472

I came up with a new approach that I'm going to try out:

Maybe a better approach would be to just submit the state to my own server, store the compressed JSON in an S3 bucket, and only send the URL to Sentry. (The compression will actually make a huge difference, because I have a lot of repeated data in my state, so it can compress from 220KB down to 5KB.) And then I can just copy that URL, and paste it into the developer console to load the state.
I could even set an expiration rule on the S3 bucket, so that old files are deleted after 3 months. I think I'm going to create an open source library that can manage all of this, specifically for Ruby on Rails. Will call it something like raven-redux-rails.

Basically I'm going stop sending any Redux state to Sentry. I'm going to add some API endpoints to my own application, and store the compressed state in my own S3 bucket. Then I'll just send that URL to Sentry. The API specification should be really easy to implement if you're already using S3 and you have authentication for admin users. Let me know if you're interested in contributing a library for other languages/frameworks, like Django, Laravel, Spring, etc.

I've started a repo here, in case anyone wants to follow along: https://github.com/FormAPI/raven-redux-rails)

from raven-for-redux.

@beaugunderson Yeah I needed to make a few changes to get it working. I ended up with a solution and have opened a PR: #95

I've tested this in development and have written some tests that are all passing, so it seems to work pretty well. I'm going to deploy it to production now and see if there are any issues.

from raven-for-redux.

I suppose I could use the stateTransformer option..

from raven-for-redux.

But also the conditions here: https://docs.sentry.io/learn/quotas/#attributes-limits seem quite limiting

from raven-for-redux.

Noting that now raven-js and @sentry/browser default to fetch() with { keepalive: true } if fetch() is available. { keepalive: true } limits POST requests to a 64k body, so overriding fetchParams with { keepalive: false } can help here too (Sentry's documented limit is 100k, and onpremise users have effectively no limit).

More info at getsentry/sentry-javascript#1464

from raven-for-redux.

I just noticed this happening while I was testing errors in development, and now I'm really worried that I've been missing a lot of error reports in production! I'm using redux-undo, so my state gets really large after only a few changes. Interestingly, compressing it with gzip turns 220kb of JSON into 5kb.

This should be a huge warning at the top of the README. I also think that raven-for-redux should be responsible for handling 413 responses, and it could retry the request without any state data. I can try to work on a PR for this. I'm also going to try to gzip the data on the client, and will see if Sentry accepts these requests.

I imagine that this issue has probably caused thousands of error reports to be silently dropped, so this is a huge problem that users should be aware of.

from raven-for-redux.

@captbaritone I just saw your workaround here: getsentry/sentry-javascript#339 (comment)
I think it would be awesome if this was part of raven-for-redux

EDIT: Ah there was a bug in your code (or sentry-javascript has changed). We need to call originalTransport.call(Raven, opts); now, otherwise this is not correct inside _makeRequest.

EDIT 2: Should also note that the maximum payload size has been increased to 200KB: https://docs.sentry.io/clients/javascript/usage/#raven-js-additional-context

from raven-for-redux.

from raven-for-redux.