test(scripts): derive the eager-closure headroom check from the report, and re-baseline the aggregate ceiling

.changeset/5924-live-headroom-invariant.md

@@ -0,0 +1,31 @@
+---
+---
+
+Build tooling and CI only — `scripts/check-eager-closure-budget.mjs`, its unit
+test, and a comment in `.github/workflows/performance-budget.yml`. Nothing ships
+from this change.
+
+The console eager-closure gate stated a binding constraint on its own
+sensitivity — the headroom above the measured payload must stay SMALLER than the
+89 KiB regression the gate exists to catch — and then checked it between two
+constants frozen in the same module. That assertion is an arithmetic fact about
+the file, true regardless of what the console weighs. The closure shrank 706,013
+gzipped bytes below the pinned baseline without the ceiling following it down,
+the live headroom reached 8.6x the regression size, and the check that was
+supposed to notice stayed green throughout: a demonstrated +158,006-byte eager
+regression, 1.7x the incident this gate was built for, passed with a green tick.
+
+`evaluateHeadroomSensitivity` now derives that headroom from the report the gate
+just read, for the aggregate ceiling and for each of the three per-chunk
+ceilings, and treats a ceiling that has drifted more than one regression above
+its own measurement as an ERROR (exit 2) rather than a size failure — it is a
+verdict about the gauge, the same asymmetry a budgeted chunk absent from the
+report already carried. The constant-vs-constant assertions stay as a secondary
+guard.
+
+The aggregate ceiling is re-baselined downward as the decision this records:
+`MAX_EAGER_CLOSURE_GZIP_BYTES` 4,086,000 to 3,345,000 over a `BASELINE` moving
+4,005,911 (`4c1623c0c`) to 3,299,898 (`48e53814e`). Headroom goes from 8.63x the
+regression size to 0.49x. Lowering a ceiling toward reality is a tightening: no
+build that passed before and measures under the new line fails after it. The
+floor is unchanged — a ceiling is never put below a measured figure.

.github/workflows/performance-budget.yml

@@ -88,9 +88,14 @@ jobs:
  # `scripts/check-eager-closure-budget.mjs` next to the reasoning that
  # produced it, so a re-baseline is one edit against a documented
  # argument rather than a number in YAML: today's measured payload plus
- # ~2% of headroom, chosen to pass on today's `main` while staying
- # NARROWER than the 89 KiB regression the gate exists to catch. It is a
- # truthful current-state ceiling, not a statement that 3.79 MB is fine.
+ # half the 89 KiB regression the gate exists to catch, which keeps the
+ # headroom NARROWER than that regression while leaving as much room for
+ # the payload to shrink as to grow. Since objectui#5924 the checker
+ # enforces that itself, against the report it just read rather than
+ # against a frozen constant, for the aggregate ceiling and each
+ # per-chunk one — a ceiling that has drifted out of range of the
+ # regression it must catch is exit 2, a verdict about the gauge. It is
+ # a truthful current-state ceiling, not a statement that 3.15 MB is fine.
  #
  # Both measurements run before either may fail the step: a run that
  # reports one number and hides the other teaches readers to distrust

scripts/__tests__/check-eager-closure-budget.test.ts

@@ -15,6 +15,7 @@ import {
  REGRESSION_THIS_GATE_MUST_CATCH_BYTES,
  SUPPORTED_REPORT_VERSION,
  evaluateClosureBudget,
+ evaluateHeadroomSensitivity,
  evaluatePerChunkBudgets,
  main,
  measureChunksByName,
@@ -89,6 +90,17 @@ describe('the ceiling itself', () => {
  expect(evaluateClosureBudget({ report: closureOf(BASELINE.gzipBytes) }).status).toBe('pass');
  });
 
+ /**
+ * ⚠️ This pair of assertions is the SECONDARY guard, and objectui#5924 is the
+ * record of what it cannot do. Both operands are literals frozen in the
+ * checker, so it is true regardless of what the console weighs: it stayed
+ * green while the closure fell ~706 KB below the pinned baseline and the live
+ * headroom reached 8.6x the regression size. It is kept because it still
+ * catches the one thing it can — an edit that raises a ceiling past the
+ * regression size, with no build in sight — and it is no longer the only
+ * check of this invariant. The live one is `evaluateHeadroomSensitivity`,
+ * exercised further down.
+ */
  it('would have failed on the regression it exists to catch', () => {
  const headroom = MAX_EAGER_CLOSURE_GZIP_BYTES - BASELINE.gzipBytes;
  expect(headroom).toBeLessThan(REGRESSION_THIS_GATE_MUST_CATCH_BYTES);
@@ -356,6 +368,185 @@ describe('per-chunk ceilings', () => {
  });
 });
 
+/**
+ * objectui#5924 — the headroom invariant, checked against the report the gate
+ * just read instead of against two literals frozen beside it.
+ *
+ * The invariant itself is old and stated in the checker's header: the headroom
+ * above the measurement must stay SMALLER than the regression the gate exists
+ * to catch, or a repeat of that regression fits inside it and passes. What was
+ * new in objectui#5924 is where it was checked. `MAX_EAGER_CLOSURE_GZIP_BYTES -
+ * BASELINE.gzipBytes < REGRESSION_...` is an arithmetic fact about the module,
+ * true forever once written, and it stayed true while the console shrank ~706
+ * KB underneath it — leaving a demonstrated +158,006-byte eager regression
+ * green, 1.7x the incident the gate was built for.
+ *
+ * These tests are about the four ceilings this file now ships (the aggregate
+ * plus the three per-chunk lines objectui#5490 added), each weighed against its
+ * own measurement in the report.
+ */
+describe('ceiling sensitivity, judged live (objectui#5924)', () => {
+ /**
+ * A v2 report totalling exactly `totalGzipBytes`, carrying the budgeted
+ * chunks at their measured sizes and the remainder of the closure as one
+ * filler chunk — the shape a real report has, where the budgeted names are a
+ * minority of the total.
+ */
+ function sensitivityReport(totalGzipBytes: number, sizes: Record<string, number> = {}) {
+ const measured: Record<string, number> = { ...PER_CHUNK_BASELINE, ...sizes };
+ const named = [
+ { fileName: 'assets/index-A.js', name: 'index', bytes: 0, gzipBytes: 25_910 },
+ ...Object.entries(measured).map(([name, gzipBytes]) => ({
+ fileName: `assets/${name}-hash.js`,
+ name,
+ bytes: 0,
+ gzipBytes,
+ })),
+ ];
+ const files = [
+ ...named,
+ {
+ fileName: 'assets/rest-of-closure.js',
+ name: 'rest-of-closure',
+ bytes: 0,
+ gzipBytes: totalGzipBytes - named.reduce((n, f) => n + f.gzipBytes, 0),
+ },
+ ];
+ return report({
+ files,
+ eagerChunkCount: files.length,
+ eagerGzipBytes: totalGzipBytes,
+ eagerRawBytes: 0,
+ });
+ }
+
+ /** The ceiling this file shipped before objectui#5924 re-baselined it. */
+ const CEILING_BEFORE_5924 = 4_086_000;
+
+ it('reds on the drift objectui#5924 recorded: 8.6x the regression above the live payload', () => {
+ const result = evaluateHeadroomSensitivity({
+ report: sensitivityReport(BASELINE.gzipBytes),
+ budgetBytes: CEILING_BEFORE_5924,
+ });
+ expect(result.status).toBe('error');
+ expect(result.blind).toEqual(['aggregate']);
+ expect(result.message).toContain('DRIFTED');
+ // The multiple, so the failure states HOW blind rather than merely that it is.
+ expect(result.message).toContain('8.63x');
+ // ...and the constant to lower, so the fix is one named edit.
+ expect(result.message).toContain('MAX_EAGER_CLOSURE_GZIP_BYTES');
+ });
+
+ it('is the check the frozen-constant assertion structurally could not be', () => {
+ // Same moment, same payload, the assertion that was supposed to guard it:
+ // the two constants of the day satisfied it comfortably, which is why the
+ // suite was green through the run above.
+ expect(CEILING_BEFORE_5924 - 4_005_911).toBeLessThan(REGRESSION_THIS_GATE_MUST_CATCH_BYTES);
+ });
+
+ it('passes on the constants and the measurement this file ships today', () => {
+ const result = evaluateHeadroomSensitivity({ report: sensitivityReport(BASELINE.gzipBytes) });
+ expect(result.status).toBe('pass');
+ expect(result.blind).toEqual([]);
+ // Every ceiling in the file is weighed, not just the aggregate one: the
+ // population objectui#5490 grew to four is the population judged here.
+ expect(result.sites.map((site) => site.key)).toEqual([
+ 'aggregate',
+ ...Object.keys(PER_CHUNK_GZIP_CEILINGS),
+ ]);
+ // A passing run still prints every measurement, so a reader watching a
+ // ceiling drift upward sees it coming rather than the day it reds.
+ expect(result.message).toContain('3222.6');
+ });
+
+ it('is exactly one regression wide, from either side of the line', () => {
+ const atTheLine = MAX_EAGER_CLOSURE_GZIP_BYTES - REGRESSION_THIS_GATE_MUST_CATCH_BYTES;
+ expect(evaluateHeadroomSensitivity({ report: sensitivityReport(atTheLine + 1) }).status).toBe(
+ 'pass',
+ );
+ expect(evaluateHeadroomSensitivity({ report: sensitivityReport(atTheLine) }).status).toBe(
+ 'error',
+ );
+ });
+
+ it('judges the per-chunk ceilings too — the population is four ceilings, not one', () => {
+ const result = evaluateHeadroomSensitivity({
+ report: sensitivityReport(BASELINE.gzipBytes, {
+ framework: PER_CHUNK_GZIP_CEILINGS.framework - REGRESSION_THIS_GATE_MUST_CATCH_BYTES,
+ }),
+ });
+ expect(result.status).toBe('error');
+ expect(result.blind).toEqual(['framework']);
+ expect(result.message).toContain("PER_CHUNK_GZIP_CEILINGS['framework']");
+ });
+
+ it('leaves a ceiling BELOW the payload to the size verdict, and says so', () => {
+ // Negative headroom is an over-budget bundle. Reporting it here as well
+ // would turn one regression into an error and teach a reader that exit 2
+ // does not mean what the file says it means.
+ const result = evaluateHeadroomSensitivity({
+ report: sensitivityReport(MAX_EAGER_CLOSURE_GZIP_BYTES + 500_000),
+ });
+ expect(result.status).toBe('pass');
+ expect(result.message).toContain('the size verdict owns this row');
+ });
+
+ it('errors when there is no report — a ceiling with no measurement is not sensitive', () => {
+ const result = evaluateHeadroomSensitivity({ report: null });
+ expect(result.status).toBe('error');
+ expect(result.message).toContain('broken gauge');
+ });
+
+ it('errors on a report it cannot trust rather than judging drift from a bad number', () => {
+ const result = evaluateHeadroomSensitivity({
+ report: { ...sensitivityReport(BASELINE.gzipBytes), reportVersion: 1 },
+ });
+ expect(result.status).toBe('error');
+ expect(result.blind).toEqual([]);
+ });
+
+ it('refuses to judge a ceiling whose chunk is absent, instead of reading it as drifted', () => {
+ // The wrong-reason trap: a budgeted chunk that is not in the report weighs
+ // zero, so its whole ceiling would look like headroom — "drifted", the
+ // right exit code for the wrong reason, on a run the per-chunk half already
+ // explains correctly.
+ const base = sensitivityReport(BASELINE.gzipBytes);
+ const files = base.files.filter((f) => f.name !== 'ui-components');
+ const result = evaluateHeadroomSensitivity({
+ report: report({
+ files,
+ eagerChunkCount: files.length,
+ eagerGzipBytes: files.reduce((n, f) => n + f.gzipBytes, 0),
+ eagerRawBytes: 0,
+ }),
+ });
+ expect(result.status).toBe('error');
+ expect(result.blind).toEqual([]);
+ expect(result.message).toContain('ui-components');
+ expect(result.message).toContain('absent');
+ });
+
+ /**
+ * The acceptance test of objectui#5924, pinned so it cannot quietly come
+ * undone. Measured on `48e53814e`: an eager `@objectstack/spec/cloud`
+ * namespace import into `apps/console/src/main.tsx` — a use the bundler
+ * cannot fold away — put 158,006 gzipped bytes into the eager closure, and
+ * removing it returned the measurement to 3,299,898 exactly, so the movement
+ * is the injection and not build noise.
+ */
+ it('the demonstrated regression: green under the old ceiling, red under the new one', () => {
+ const INJECTED = BASELINE.gzipBytes + 158_006;
+ // 1.7x the incident this gate was built to catch.
+ expect(INJECTED - BASELINE.gzipBytes).toBeGreaterThan(REGRESSION_THIS_GATE_MUST_CATCH_BYTES);
+
+ const injected = sensitivityReport(INJECTED);
+ expect(
+ evaluateClosureBudget({ report: injected, budgetBytes: CEILING_BEFORE_5924 }).status,
+ ).toBe('pass');
+ expect(evaluateClosureBudget({ report: injected }).status).toBe('fail');
+ });
+});
+
 describe('renderTopChunks', () => {
  it('names the biggest eager chunks so a failure has suspects', () => {
  const lines = renderTopChunks(report(), 2).split('\n');
@@ -401,8 +592,8 @@ describe('main', () => {
  const { code, outputs } = run(budgeted());
  expect(code).toBe(0);
  expect(outputs.closure_status).toBe('pass');
- expect(outputs.closure_chunks).toBe('4');
- expect(outputs.closure_gzip_kb).toBe('1814.3');
+ expect(outputs.closure_chunks).toBe('5');
+ expect(outputs.closure_gzip_kb).toBe('3222.6');
  });
 
  it('exits 1 — a verdict about the BUNDLE — when over budget', () => {
@@ -438,10 +629,21 @@ describe('main', () => {
  expect(outputs.closure_gzip_kb).not.toBe('');
  });
 
- /** A v2 report at the measured per-chunk sizes, well inside the aggregate. */
- function budgeted(sizes: Record<string, number> = {}) {
+ /**
+ * A v2 report at the measured per-chunk sizes, totalling `BASELINE.gzipBytes`
+ * plus `totalDelta`.
+ *
+ * ⚠️ The filler chunk is not padding. Before objectui#5924 this fixture
+ * carried only the budgeted names, so its total was ~1.8 MB against a 3.3 MB
+ * ceiling — a shape `main` now (correctly) calls a BLIND ceiling and exits 2
+ * on. A report whose total sits far below the aggregate line is not a
+ * within-budget bundle to be asserted `pass`; it is the defect this card
+ * fixed. So the fixture carries the rest of the closure, as a real report
+ * does, and `totalDelta` is how a test moves the total on purpose.
+ */
+ function budgeted(sizes: Record<string, number> = {}, totalDelta = 0) {
  const measured: Record<string, number> = { ...PER_CHUNK_BASELINE, ...sizes };
- const files = [
+ const named = [
  { fileName: 'assets/index-A.js', name: 'index', bytes: 0, gzipBytes: 25_910 },
  ...Object.entries(measured).map(([name, gzipBytes]) => ({
  fileName: `assets/${name}-hash.js`,
@@ -450,6 +652,19 @@ describe('main', () => {
  gzipBytes,
  })),
  ];
+ // Derived from the BASELINE sizes, not from `named`, so an override in
+ // `sizes` moves the total the way a real chunk growing would.
+ const baselineNamed =
+ 25_910 + Object.values(PER_CHUNK_BASELINE).reduce((n, bytes) => n + bytes, 0);
+ const files = [
+ ...named,
+ {
+ fileName: 'assets/rest-of-closure.js',
+ name: 'rest-of-closure',
+ bytes: 0,
+ gzipBytes: BASELINE.gzipBytes - baselineNamed + totalDelta,
+ },
+ ];
  return report({
  files,
  eagerChunkCount: files.length,
@@ -497,6 +712,35 @@ describe('main', () => {
  expect(outputs.closure_chunk_status).toBe('error');
  });
 
+ /**
+ * objectui#5924: the run that used to be the file's blind spot. Both size
+ * halves are delighted — nothing is over any line — and the gate still has to
+ * stop, because neither of those green ticks means anything at this distance.
+ */
+ it('exits 2 when a ceiling has drifted out of range of the regression it must catch', () => {
+ const { code, outputs } = run(budgeted({}, -REGRESSION_THIS_GATE_MUST_CATCH_BYTES));
+ expect(code).toBe(2);
+ expect(outputs.closure_status).toBe('pass');
+ expect(outputs.closure_chunk_status).toBe('pass');
+ expect(outputs.closure_headroom_status).toBe('error');
+ });
+
+ /**
+ * The ordering objectui#5490 established over two halves, held over three:
+ * `error` outranks `fail`, whichever half noticed. `performance-budget.yml`
+ * maps exit 2 to `budget_status=error` and any other non-zero to `fail`, so
+ * collapsing this to 1 would report a gauge that cannot be trusted as a size
+ * regression.
+ */
+ it('reports the GAUGE verdict when one ceiling is blind and another is over', () => {
+ const { code, outputs } = run(
+ budgeted({ 'vendor-objectstack': PER_CHUNK_GZIP_CEILINGS['vendor-objectstack'] + 1 }, -200_000),
+ );
+ expect(code).toBe(2);
+ expect(outputs.closure_chunk_status).toBe('fail');
+ expect(outputs.closure_headroom_status).toBe('error');
+ });
+
  it('exits 2 on a report from a build that predates per-chunk names', () => {
  const base = budgeted();
  const { code, outputs } = run({ ...base, reportVersion: 1 });