Skip to content

docs(headless): document v4 report rate semantics - #1763

Merged
Astro-Han merged 1 commit into
mainfrom
docs/headless-report-rate-semantics
Aug 1, 2026
Merged

docs(headless): document v4 report rate semantics#1763
Astro-Han merged 1 commit into
mainfrom
docs/headless-report-rate-semantics

Conversation

@Astro-Han

Copy link
Copy Markdown
Contributor

Summary

Documents the report-semantics change that #1758 (5c2bbc6db) introduced but did not disclose: schema v4's top-level effectiveness pass rates are arm-local (each arm's passes over its own valid cells), not the v3 shared paired denominator. Runs with unilateral infra gaps therefore read differently between v3 and v4 reports.

  • field-level comments on HarnessAbArmEffectiveness.passRate, candidateMinusBaseline vs pairedCandidateMinusBaseline, and the nonBudgetConditional section (diagnostic disclosure, never the headline)
  • README's report paragraph updated (it still described "schema v3")

No behavior change; comments and README only.

Verification

  • packages/headless suite: 1456 pass / 0 fail (report tests confirm no code drift)
  • format/lint clean

Review focus

Wording accuracy against buildHarnessAbReport: arm-local numerators/denominators at harness-ab-report.ts:135-146, paired delta at summary.passRateDelta.

@Astro-Han
Astro-Han merged commit 55cdd54 into mainAug 1, 2026
3 checks passed
@Astro-Han
Astro-Han deleted the docs/headless-report-rate-semantics branch August 1, 2026 06:37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@Astro-Han