Document that there's a limit in the size of the data that can be sent

[cmdelatorre]

As long as I understand, after reading #58 and other issues, raven-js uses HTTP GET requests to send data to Sentry. Apparently (and unlucky for me) it is a design decision related to the difficulties of handling CORS.

In any case, the fact is that there's a strict limit on the size of the error reports, that depends on the max url size supported by the Sentry web server. As I'm using the getsentry service, there's nothing I can do about it.

So, careless of how much I'd love to be able to POST longer reports to Sentry, I'd strongly recommend that you document this issue properly. I consider it to be very important and should be somewhere in the official documentation of the API.

Finally, thanks a lot for this great product.

[mattrobenolt]

This has changed drastically in 2.0.0 since we now send data via POST.

But, there is still a limitation depending on your server. If you're using on-premise, it's impossible to determine, but if you're talking to our servers (app.getsentry.com), our limit as of now is 100KB for the body.

If the message is too large, it'll be rejected.

@benvinegar We should add this to docs now.

[jakubzitny]

Hey guys, have you considered adding a check for the Sentry message size limit into Raven?

What do you think would be the best way to check / truncate the extra data that's over the limit?

(cc @benvinegar @mattrobenolt)

[mattrobenolt]

sentry.io has a limit that is effectively arbitrary for ourselves. This limit doesn't apply and may different for anyone running Sentry on-premise. Our limit as of today is 100k, but it's possible that in the future we may increase this size.

Now, there's no way to communicate this back to the server until it fails to make the request and is rejected. In theory, we could change the error that's kicked back to be JSON and be something like, {"error":413, "limit":100}, then trim and retry with that information it got. But this would be sorta specific to our implementation since this happens at our load balancer and not in the Sentry server itself.

it's also worth noting that we don't deal with this in other clients nearly as often since they all gzip their payloads before transmitting, so it's very rare to cross our 100k limit. But for JavaScript, we're not able to do this easily without bloating raven-js with a gzip library. (Ironic that we can't easily gzip from a browser, huh?)

[jakubzitny]

Thanks for the reply @mattrobenolt.

I understand the situation, but what if this was configurable? When configuring or instantiating the Raven client I'd add a parameter specifying the limit for one event and the client would automatically trim the contents somehow. Would it make sense?

(cc @vojtatranta)

[benvinegar]

There's also not a really easy way to say "limit this to 100kb". Users can pass extra values with arbitrarily deep objects and sub-objects, so we'd need an algorithm that basically traverses the entire tree and decides where to trim large items. And there's a ton of tricky cases to handle, e.g. how does it know that an array of 100 small items is less bytes than an array of 10 items with large items? This algorithm could become really complex, really fast.

We've experimented with payload size, and found that for a typical maxed out stack trace, and 100 breadcrumbs, the payload is roughly ~15 KB. That's not a lot of bytes for a ton of information. Users only really hit the 100kb limit if they add arbitrarily large arrays/objects to extra manually.

I think if anything, we should add a note to the extra config warning that there is an upper size limit, and users shouldn't indiscriminately add complex objects of unknown size.

[jakubzitny]

Yeah that's true.

Actually, the large objects in extra is exactly our case. We're attaching some logs (little bit similar to breadcrumbs), that can really be "misused" by a developer and filled with unneccessarily big objects. That's why we're contemplating about the limits there and what'd be the best way to trim it.

[davishmcclurg]

I just ran into this with an error that had a bunch of breadcrumbs. It would be nice if the client would re-send the error without breadcrumbs or extra data so that it doesn't disappear completely.

[davidfurlong]

With redux sentry middleware our entire redux state gets sent in extra. Now we're getting this. Any idea of a current estimate on max size? Looks like one request was around 240KB