[minor] Chart performance per release, and show it in the README

[matt-edmondson]

Summary

The README now opens on a picture of what each release did to allocation and speed, drawn from numbers this repository keeps rather than from a run someone remembered to do.

Three pieces:

Piece	Job
PreciseNumber.Benchmarks/BaselineBenchmarks.cs	A fixed integer workload touching none of this library — the reading of the machine that makes separate jobs comparable.
scripts/benchmark-history.cs	A .NET 10 file-based app. ingest reads BenchmarkDotNet's JSON into a committed history keyed by version, render draws it as SVG, baseline reports the reference time. Run with dotnet run scripts/benchmark-history.cs -- …, so it needs no project and nothing installed beyond the SDK that already builds the library.
.github/workflows/benchmark-history.yml	Measures a published release and adds a point, or backfills a list of refs on workflow_dispatch.

This is deliberately separate from benchmarks.yml, which exists to get a full ad-hoc run on demand and publishes nothing.

The awkward part, addressed rather than hidden

A benchmark chart across releases is mostly a chart of CI runners. Each release is measured in its own job, and the difference between the hosts a job can land on is larger than most releases are — the one-off comparison behind #76 had to put both versions in a single job for exactly this reason.

Two things deal with it:

• Allocation is charted because it is exact. The same code allocates the same bytes on any machine, so a step in the top row is always a real change.
• Times are divided by the reference workload measured in the same job. That cancels most of the difference between machines, and the README says plainly that what is left is indicative — a small wobble between two releases is more likely the runner than the library.

A backfill measures every ref in one job for the same reason: points gathered on one host are comparable as they stand, and the reference reading ties that job to later ones.

Seeded with real data

The chart ships with 1.9.0 and 2.0.0–2.0.4 already measured, so the README works the moment this merges rather than showing a broken image until the first release.

Those six points were measured in a dev container, not on a GitHub runner — each entry records the CPU it was measured on, and ingest replaces an entry for the same version, so running the backfill from the Actions tab cleanly overwrites them with runner-measured points whenever you want. The dispatch defaults are already set to that ref list.

It reproduces the #70 result independently, which is a decent check on the pipeline: every allocation drop at 2.0.0 is exactly 40 bytes — the object header — and ToDouble falls from 0.045 to 0.010 of the reference, the same ~79% the one-off comparison on #72 measured.

README

The existing Performance section moved up to sit after Features, rather than a second ## Performance heading being added, which would have taken the #performance anchor and broken the contents link. Light and dark charts are swapped with <picture> + prefers-color-scheme.

The tool was Python first, and that was wrong

Worth recording, since the same tool is meant to be copied into Semantics and SignificantNumber.

It was written in Python because that was quick and needs no pip install on a runner — a reason about writing it rather than living with it. There is no other .py file in any of the three repositories, PowerShell is what they use for scripting, and nothing in the repo's tooling covered it.

SonarCloud then failed the Quality Gate on it: a S2583 bug (a condition always evaluating to true) took the reliability rating to C, plus two cognitive-complexity findings and an argument of the wrong type passed to escape.

As a file-based app it is covered by the build instead, and the first compile failed on CA1305, CA1502, CA1505, CA1506, IL2026, IL3050 and SYSLIB1045. Most are fixed rather than suppressed — cultures are explicit, the regex is source-generated, and the work moved out of top-level statements into a class so each method is judged separately. Two of those are the same findings Sonar made about the Python, caught before the push instead of after.

The port is verified by output: re-ingesting the same reports and re-rendering leaves the history and both SVGs byte for byte identical. Getting there caught one real difference — parameters were coming out in ordinal order, which reads 200, 30, 8, where BenchmarkDotNet declares them 8, 30, 200 and the Digits axis is meant to be read across in order.

Five bugs the testing caught

Each would have failed silently:

1. git diff --quiet does not see an untracked file, so the very first run would have committed nothing and still reported success — the same failure mode as the artifact-path bug in [patch] Write benchmark results where the workflow looks for them #73. Now staged first, then compared against the index.
2. git show -s --format=%cs on an annotated tag prints the whole tag object, so every tagged entry recorded the tagger message where its date belonged. Caught by reading the real backfill output.
3. git rev-parse --short v2.0.0 resolves to the tag object (0dbce0a), not the commit (da3aac6).
4. A release event checks out the tag, so committing results from that checkout asks the default branch to move backwards. Both paths now check out the default branch and measure their ref through a worktree.
5. ktsu.Sdk regenerates .gitignore on every build, so ignore entries added for the workflow's scratch directories did not survive a single dotnet build. Scratch output goes to BenchmarkDotNet.Artifacts/ instead, already ignored, and worktrees are created under RUNNER_TEMP, outside the repository where the commit step cannot see them.

Verified

• Solution builds with 0 warnings, 0 errors; all 266 tests pass.
• The chart palette was checked with a CVD validator in both light and dark modes — all checks pass, worst adjacent ΔE 24.7 / 26.8.
• Both SVGs were rendered to PNG and inspected. That is what caught fast benchmarks collapsing to 0.01×: the set spans three orders of magnitude, so ratios now use significant figures.
• The workflow's steps were exercised against real artifacts — the baseline subcommand returns a value, ingest is idempotent, and the staged-diff check detects new files.

Note on scope

You asked for this across Semantics and SignificantNumber too. Neither has a benchmark project at all, so each needs a suite written before it can be charted. This PR establishes the mechanism on the repo that already has one; the tool and workflow are written to port with only the headline list changing.

🤖 Generated with Claude Code

https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud