Capture request/response headers for outgoing requests in Replay

[mydea]

Similar to #7103, we may also want to capture the headers sent with requests.

Questions:

• This should be PII-stripped automatically server side, I assume cc @cmanallen
  • If we provide a POJO of the headers, will each header be stripped separately?
• Should we define a white list of headers we want to capture, or just all headers?
• Should we capture headers for request AND response, or just for responses? (Note: For XHR, request headers are doable but a bit tricky)
• A lot of the headers can be pretty static for each outgoing request, so there may be little value in capturing all of them.

Proposed solution

I propose to define a include-list of headers to capture, and allow users to configure this if they want to capture something specific:

new Replay({
 networkBreadcrumbs: {
 headers: ['My-Custom-Header']
 }
}

The list could look like:

• Content-Length
• Content-Type
• Accept

AWS specific headers:

• X-Amzn-Trace-Id

• More to add ??

Example breadcrumb:

 {
 data: {
 url: 'my-url.com',
 method: 'GET',
 response: {
 headers: {
 'Accept': 'application'json',
 'Authentication': '[redacted]'
 }
 }
}

[cmanallen]

This should be PII-stripped automatically server side

Yes.

Should we define a white list of headers we want to capture, or just all headers

We definitely need to not-send and/or scrub session tokens. That's a huge security liability. We should probably expose user configuration to scrub their sensitive headers in addition to whatever system we use to scrub heuristically. I would definitely remove any header whose keys contains auth or token.

A lot of the headers can be pretty static for each outgoing request, so there may be little value in capturing all of them.

A user defined whitelist of headers solves a lot of problems... Should a user choose to collect a static header then it is what it is. The cost is not so large.

[mydea]

Sounds good to me!

I wonder if we go with a include-list of headers, should we still enforce to remove headers including e.g. auth or token? I guess we could start out with doing that and always revisit later if we find out this does something unexpected.

[billyvg]

I wonder if we go with a include-list of headers, should we still enforce to remove headers including e.g. auth or token? I guess we could start out with doing that and always revisit later if we find out this does something unexpected.

I think we should, it should be ok as long as we clearly document it.

For API, maybe instead of networkBreadcrumbs -> network? or networkRequests?