Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions AGENTS.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -335,6 +335,76 @@ When the user says `open PR`, `create PR`, or `publish PR` without also requesti
- For pull requests that touch ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output, complete the clinical governance preflight in `.github/pull_request_template.md`.
- Track known verification debts and staged process improvements in `docs/process-hardening.md` instead of relying on chat-only memory.

## Do not pay twice for the verdict GitHub is about to reach

`check:gate-manifest` enforces a one-way invariant: CI never runs LESS of the local
`verify:cheap` static set than the local chain does. Read that the other way and it says
something uncomfortable — **every local run of a gate in that chain is work GitHub is
about to repeat.** `gate-receipts.mjs` removed the local-versus-local duplication (the
same gate twice on unchanged content); it explicitly cannot touch this one, because CI
must never reuse a receipt.

That does not make the local run waste. It is a **bet**: a local run that fails saves a CI
round trip, and a red or superseded push is expensive here (~40% of PR CI runs measured
2026-07-30 were cancellations). A local run that passes bought nothing the CI run would not
have established. So the question is never "local or CI" in the abstract, it is:

> **Is this gate, on this kind of change, still catching anything?**

**The rule: run an expensive local gate only while it is still earning its runtime, and
never re-derive a verdict that already exists.** Before running `lint`, `typecheck`,
`test`, `verify:cheap`, or `verify:pr-local`, consult the arbiter and quote its verdict:

```bash
npm run arbiter -- <gate> # RUN / DEFER / PROVEN, with its evidence
npm run arbiter:status # the yield ledger and the duplication bill so far
```

It weighs three inputs, none of them hard-coded, so the answer moves as the repo moves:

1. **CI coverage**, derived live from `package.json` + `.github/workflows/ci.yml`, and
evaluated **for this change** — the step's own `if:` and its job's `if:` are checked
against the current change scope, because a step's presence in the YAML is not
coverage. `lint` and `typecheck` are step-conditional on `static_heavy_changed` and
`test:coverage` is job-conditional on `coverage_changed`, so a docs-only change is
covered by none of them. A gate CI does not re-run is never deferrable — local is the
only gate there is. Delete the CI job and the arbiter stops deferring to it the same day.
2. **Observed yield**, a rolling per-gate, per-change-class window of local outcomes that
the gate wrappers record automatically. A gate that has caught nothing across a full
clean window on this class of change has stopped earning its runtime. The **first catch
resets the window** and the gate runs locally again, so the loop re-arms itself instead
of decaying toward "never check anything".
3. **Content identity** — a verdict GitHub already reached on exactly this content
(recorded with `npm run arbiter -- record-ci <sha> <gates…>` when a session observes CI
go green) is not re-derived locally. This is the common repetition: CI goes green on a
branch head, and a later session runs the whole suite again on that same head. Name the
gates CI actually ran — the command refuses a bare invocation rather than turning one
observed job into proof for every gate.

The window is per change class because the classes are not the same bet: docs-only clears
in 3 clean runs, source in 12, and **db, RAG, dependency, container, workflow, UI and
unrecognised scope never defer at all**, however clean the history — the same fail-closed
routing CI itself uses, not a second risk model.

Non-negotiable boundaries, all of them the conservative direction:

- **Fail open.** Missing data, unreadable CI, an unknown change class, a git failure — every
one of them runs the gate. A bug in the arbiter costs a redundant run, never a skipped one.
- **CI is never advised by it.** `CI` being set disables the arbiter outright. GitHub stays
the authoritative merge gate and nothing computed locally may influence what it runs.
- **Advisory by default.** A `DEFER` or `PROVEN` verdict is a recommendation printed with
its evidence; the wrappers act on it only under `GATE_ARBITER=enforce`. Silently skipping
a gate a human typed is exactly the failure the evidence rules exist to prevent.
- **A focused run is not full-suite evidence.** A narrowed Vitest invocation records under
its own identity, so a clean run of single-file tests can never let the whole suite defer.
- **A deferred gate is not a passed gate.** Report it as "deferred to CI — <gate> has caught
nothing in N consecutive <class> runs", never as green, and never alongside a claim that
the gate ran. The same applies to `PROVEN`: say "reused receipt" or "CI-proven at `<sha>`".

This does not license skipping verification. It licenses not buying the _same_ verdict
twice. The smallest-correct-gate rule above still decides which gate is right; the arbiter
only decides whether that gate has anything left to tell you before you push.

<!-- END:process-hardening -->

<!-- BEGIN:page-and-button-wiring -->
Expand Down
72 changes: 72 additions & 0 deletions docs/process-hardening.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -81,6 +81,78 @@ what it is scoped to verify, and `ci-change-scope.mjs` already keeps that scoped
not waste either — it is what stops a red push costing a full CI cycle plus a fix round. The waste
was only ever the _repeat_, and that is what is now gone.

## Gate arbitration: stop paying for a verdict GitHub is about to reach (2026-08-21)

The receipts section above closes by saying the local run "is not waste either — it is what stops a
red push costing a full CI cycle". True, and incomplete. `check:gate-manifest` enforces that CI never
runs less of the local `verify:cheap` static set than the local chain does, which read the other way
says **every local run of a gate in that chain is work GitHub is about to repeat**. Receipts cannot
touch that duplication by design: `receiptsEnabled()` is false whenever `CI` is set.

So the local run is a bet, not a certainty. It pays when it fails (a red push costs a CI cycle plus a
fix round, and ~40% of PR CI runs measured 2026-07-30 were cancellations); it pays nothing when it
passes. The bet's value is therefore not fixed — it decays as a gate stops catching things on a given
class of change, and it recovers the moment the gate catches something again. Nothing measured that,
so the decision was made from habit in both directions: running the full suite on a docs typo, and
skipping it on a change that deserved it.

`scripts/gate-arbiter.mjs` measures it. Three inputs, none hard-coded:

- **CI coverage**, parsed live from `package.json` + `.github/workflows/ci.yml` using the same
field-anchored `run:` regex as `check-gate-manifest.mjs` (the two must agree — a looser parse here
would defer to a job the manifest check knows does not exist). Resolved by the gate's own name, its
declared CI equivalent (`test` → `test:coverage`), or a CI-invoked aggregate whose package.json body
contains it — and then **evaluated against the current change scope**. A step's presence in the YAML
is not coverage: `lint` and `typecheck` carry a step-level `if: needs.changes.outputs.static_heavy_changed`,
and the `coverage` job is gated on `coverage_changed`, so a docs-only change is covered by none of
the three. A name-only scan reported all three covered, which under `GATE_ARBITER=enforce` produced
the one outcome the module exists to prevent — local gate deferred, CI gate skipped, no verdict
anywhere. Raised as P1 by Codex review on PR #2245 and pinned by `tests/gate-arbiter.test.ts`.
Conditions that are not change-scope flags (draft state, event name) cannot be evaluated from a
worktree; they are reported as assumed preconditions with the decision rather than silently taken as
true. **A gate CI does not re-run for this change is never deferrable.**
- **Observed yield**, a rolling window (40 observations) keyed by `(gate, change class)`, recorded by
`run-heavy.mjs` and `run-vitest.mjs` after every arbitrated run. Recording is pure observation and
never alters the run. An admission-busy exit (75) is not a verdict and is not recorded, so lock
contention can neither manufacture a clean window nor keep a healthy gate running forever.
- **Content identity**, via `record-ci <sha>`: a clean worktree plus an empty `git diff <sha> HEAD`
proves the content GitHub judged is the content in front of us. Both halves are required — a clean
tree alone does not prove HEAD has not moved, and a matching diff alone cannot see an uncommitted
edit. Reading GitHub is provider-backed, so the arbiter never reaches for it; the session that
already looked at CI passes what it saw.

Change class comes from `scripts/ci-change-scope.mjs` — the classifier CI itself uses to route jobs —
rather than a second risk model, so the arbiter and CI cannot drift into two opinions about what a
path means. Clean-window sizes: `docs` 3, `source` 12. Every other class (`db`, `rag`, `deps`,
`container`, `workflow`, `ui`, `unknown`) is absent from the window map and never defers at any length.
`tests/gate-arbiter.test.ts` pins that absence, so adding a risky class to the deferrable set fails.

Boundaries, each of them a test in `tests/gate-arbiter.test.ts`:

- **Fail open.** Unreadable CI, unknown class, missing observations, git failure — all run the gate.
- **CI never consults it.** `arbiterMode()` returns disabled whenever `CI` is set.
- **Advisory by default.** The wrappers act on a deferral only under `GATE_ARBITER=enforce`; a gate a
human typed still runs. `GATE_ARBITER=off` disables it entirely.
- **The first catch re-arms the window**, so a gate that starts failing again is never left deferred
because it had a long clean run beforehand.
- **A narrowed Vitest run records under its own identity** (`vitest(selected)`), so a clean history of
focused runs can never satisfy the full suite's window.
- **`record-ci` requires an explicit gate list**, and rejects a SHA that does not resolve here, so one
observed green job cannot become stored proof for every arbitrated gate.
- **Observations are re-read immediately before the write**, so two gates finishing together cannot
drop a catch — the unsafe direction, since a lost catch leaves a failing gate deferred.
- **A deferred gate is not a passed gate.** The verdict prints that sentence; report it as "deferred
to CI", never as green.

`npm run arbiter -- <gate>` gives the verdict and its evidence; `npm run arbiter:status` shows the
yield ledger and the accumulated duplication bill; `npm run arbiter:clear` empties it. The ledger sits
beside the receipt store under `node_modules/.cache/`, so it is per-worktree, never committed, and
destroyed by `npm ci`.

**What this does not do.** It does not reduce GitHub's work, weaken any required check, or change
which gate is the smallest correct one for a diff. It decides only whether that gate still has
anything left to tell you before you push.

## Multi-worktree reconciliation hardening (2026-07-23)

The cloud-chat reconciliation postmortem and complete issue/fix matrix are in
Expand Down
3 changes: 2 additions & 1 deletion docs/scripts-index.md
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
# Scripts index

Curated map of `scripts/` (254 files) and the `package.json` script surface (255 entries),
Curated map of `scripts/` (255 files) and the `package.json` script surface (258 entries),
grouped by purpose. This is orientation, not an exhaustive per-file listing — the authoritative
command list is `package.json`, and `npm run docs:check-scripts` verifies every `npm run <x>`
referenced in docs resolves to a real script. `npm run docs:update` refreshes the exact counts above.
Expand All@@ -19,6 +19,7 @@ migration has shipped (see `docs/maturity-backlog-workorders.md` L1).
| Script | Role |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `run-heavy.mjs` | Acquires shared/exclusive cross-worktree leases (`test-run-lock.mjs`) so focused checks can overlap safely |
| `gate-arbiter.mjs` | Decides whether an expensive local gate still earns its runtime, from live CI coverage plus a rolling per-gate/per-change-class yield window (advisory; never in CI) |
| `gate-receipts.mjs` | Content-addressed memoisation of `lint`/`typecheck`/Vitest so an unchanged gate is never rerun locally (never in CI) |
| `run-tsx.mjs`, `run-vitest.mjs`, `run-playwright.mjs`, `run-eval-safe.mjs` | Typed/test/e2e/eval entrypoint wrappers |
| `dev-free-port.mjs`, `ensure-local-server.mjs` | Project-stable localhost port selection + background server ensure |
Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -255,6 +255,9 @@
"check:local-presence": "node scripts/check-local-presence.mjs",
"receipts": "node scripts/gate-receipts.mjs status",
"receipts:clear": "node scripts/gate-receipts.mjs clear",
"arbiter": "node scripts/gate-arbiter.mjs",
"arbiter:status": "node scripts/gate-arbiter.mjs status",
"arbiter:clear": "node scripts/gate-arbiter.mjs clear",
"check:drift": "node scripts/run-tsx.mjs scripts/check-drift.ts",
"check:migration-history": "node scripts/run-tsx.mjs scripts/check-migration-history-alignment.ts",
"drift:manifest": "node scripts/run-tsx.mjs scripts/generate-drift-manifest.ts",
Expand Down
Loading
Loading