Replay SDK overhead measurement

[bruno-garcia]

Problem Statement

It's very understandable that the first question people have about the Replay SDK is "how heavy is it". The idea of tracking all web page changes already give the idea of a lot of heavy work. For that reason, we need some automation to track the overhead of the SDK.

Solution Brainstorm

Overhead measurement is a complex topic. A lot of different metrics could be used: resources such as CPU/memory as well as Web Vitals.

The goal:

1. Provide a simple overhead metric, to set expectations with users of what this might affect their website.
2. A way to avoid regressions as large differences can bring attention to a PR before it gets merged or a release is shipped

This was done for Mobile: https://github.com/getsentry/action-app-sdk-overhead-metrics
To measure the difference in size (which we already have in this repository) and the startup time difference, which is key in Mobile.

[github-actions]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

[bruno-garcia]

Assigned to myself for updates but @vaind is helping out here

[vaind]

#6611 added the tooling to produce and add some metrics to PRs as well as one scenario to measure that. The scenario added is animation (position update) heavy and evaluates performance of DOM update watcher, a.k.a Mutation Observer. This was chosen as the first scenario with the expectation that it would be the heaviest operation.
On the other hand, there are other "observers", such as mouse movements, interactions, scrolling, resizing and input which are currently not covered. Adding performance measurement scenarios for these is something to be considered in the future.