Skip to content

measure-stall-guard-headroom joins runner observations to sites by step NAME alone, so two guarded steps sharing a name in different jobs would swap headroom readings #13121

Description

@os-elon

Observation only, found while working #12959 (the census sibling note). Latent today, not a live wrong reading — filing it because the failure is silent and the tree is one duplicated step name away from it.

What it does

scripts/measure-stall-guard-headroom.mjs enumerates guarded sites by importing the gate's scan, then joins GitHub's runner timestamps onto them. The join key is the step name, and nothing else:

// judgeHeadroom(), ~line 178constbyStep=newMap();for(constoofseen){constlist=byStep.get(o.step)??[];list.push(o);byStep.set(o.step,list);}
...
constobs=byStep.get(site.step)??[];

and the same at the collection end, const wanted = new Set(swept.sites.map((s) => s.step));.

A site is really identified by the triple (file, job, step) — the tool knows this, and prints all three in every one of its own output lines (never observed: FILE job JOB step STEP). It just does not join on them.

Why it is silent

If two guarded steps in different jobs ever carry the same name:, their observations merge into one bucket and each site is then judged against obs.reduce(worst) over the union. The worst p + s of one job gets attributed to a site in another job. There is no collision detection, no warning, and the output shape is identical to a correct run — the tool would report a confident headroom verdict for a step it never measured. The direction is not even consistently conservative: it can also make a tight site look fine if the co-named step is the faster one and the observation sets are asymmetric.

Today's population — why this is latent and not live

All 7 guard-wrapped steps currently carry distinct names:

ci.yml test Run this shard's tests
ci.yml temporal-conformance Run driver-sql suite against both live servers
ci.yml temporal-conformance Run the non-SQL temporal backends under the skewed process zone
ci.yml dogfood Boot example apps and exercise real user flows
coverage-nightly.yml coverage Generate coverage report
rerun-safety-nightly.yml rerun-safety Test suite — pass 1
rerun-safety-nightly.yml rerun-safety Test suite — pass 2 (same working tree)

So no reading is wrong right now. But two of the five jobs already hold a sibling pair, and the two rerun-safety names differ only by a trailing qualifier — the shape where someone copies a step and trims the name is exactly how this arrives.

Same defect class as the card it was found under

#12959 is the same mistake one level up: an identifier that is unique only within a scope, used as if it were global. The fix landing there keys sibling grouping on (file, job id) rather than the job id alone, precisely because this tree already reuses three job ids across files (publish, patrol, registry-canary). This tool has the identical exposure on the step axis.

Suggested shape

Join on the same triple the tool already prints. A collision that cannot be resolved should REFUSE rather than pick one, matching the non-vacuity discipline the gate family already follows — a merged bucket is a measurement of something other than what the row claims.

No assignee — recording only.

Generated by Claude Code

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions