SPMI: Use diffed bytes for jit-analyze and print more info

[jakobbotsch]

Fix the misleading total bytes displayed when diffing with SPMI. If one
of the JITs encountered missing data, the code bytes would not be
included, while the other JIT could still have it included if it did not
encounter missing data.

Also add more information about missing SPMI data/successful replays
printed. For replays that is useful to be able to gauge whether there is
still ok coverage after a large change. For diffs, we print a warning
for missing SPMI data that the diff summary may be misleading.

Example for a successful replay:
Clean SuperPMI replay (219868 contexts processed)

Example for a replay with missing data:
SuperPMI encountered missing data for 6 out of 27272 contexts

Example warning printed:
Warning: SuperPMI encountered missing data during the diff. The diff summary printed above may be misleading.
Missing with base JIT: 0. Missing with diff JIT: 6. Total contexts: 27272.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix the misleading total bytes displayed when diffing with SPMI. If one
of the JITs encountered missing data, the code bytes would not be
included, while the other JIT could still have it included if it did not
encounter missing data.

Also add more information about missing SPMI data/successful replays
printed. For replays that is useful to be able to gauge whether there is
still ok coverage after a large change. For diffs, we print a warning
for missing SPMI data that the diff summary may be misleading.

Example for a successful replay:
Clean SuperPMI replay (219868 contexts processed)

Example for a replay with missing data:
SuperPMI encountered missing data for 6 out of 27272 contexts

Example warning printed:
Warning: SuperPMI encountered missing data during the diff. The diff
summary printed above may be misleading.
Missing with base JIT: 0. Missing with diff JIT: 6. Total contexts:
27272.

Author:	jakobbotsch
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[jakobbotsch]

@sandreenko for your PR this prints the following to console now:

Summary of CodeSizediffs:
(Lower is better)
Total bytes of base: 7170863 (overridden on cmd)
Total bytes of diff: 7170866 (overridden on cmd)
Total bytes of delta: 3 (0.00% of base)
diff is a regression.
relative diff is a regression.
Top file regressions (bytes):3:11295.dasm (2.86% of base)
1 total files withCodeSize differences (0 improved, 1 regressed), 1 unchanged.
Top method regressions (bytes):3 ( 2.86% of base) :11295.dasm -System.TimeSpan:Negate():System.TimeSpan:thisTop method regressions (percentages):3 ( 2.86% of base) :11295.dasm -System.TimeSpan:Negate():System.TimeSpan:this1 total methods withCodeSize differences (0 improved, 1 regressed), 1 unchanged.
--------------------------------------------------------------------------------Warning:SuperPMI encountered missing data during the diff. The diff summary printed above may be misleading.
Missingwith base JIT:0. Missingwith diff JIT:6. Totalcontexts: 27272.

[jakobbotsch]

cc @dotnet/jit-contrib

[BruceForstall]

LGTM

[BruceForstall]

@jakobbotsch It looks like superpmi.py will use the new "override base/diff metric" logic if the --metrics option is not used, so we end up using the default CodeSize metric. What if I use --metrics CodeSize,PerfScore? In that case, we could override just the CodeSize metrics, given a more general --override-total-base-metric argument to jit-analyze that takes <metric>,<value> pairs (say). I guess we don't have the PerfScore override until something like #52877 is implemented.

[jakobbotsch]

I agree that sounds like a fine way to do it. FWIW, my envisioning was to eventually pass the base and diff summary metrics files themselves (instead of just the numbers of the code size metric) to jit-analyze and have jit-analyze use directly numbers from there to present other metrics as well (e.g. perf score once we have a good way of getting it (#52877), memory allocations).