ci: give the alignment audit its own lane by Astro-Han · Pull Request #1974 · apache/maka

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han · 2026-08-03T09:08:11Z

Summary

The CDP alignment auditor ran as a final step on e2e_shard 1, sharing that runner to reuse its built renderer. Playwright itself shards evenly — a recent run measured 118s on shard 1 against 120s on shard 2 — but the 51s audit landed on one side only, making shard 1 the longer of the two for work Playwright has no data dependency on.

This moves the audit into its own alignment_audit job behind the same needs.changes.outputs.e2e gate. That gate already covers both surfaces; ci-test-plan.mjs describes its own output as "Electron E2E + alignment audit" and routes scripts/audit-alignment.mjs through E2E_DRIVING_SCRIPTS, so the selection logic needs no change.

The new lane is also added to the aggregate e2e check, so splitting it out does not quietly drop it from the required gate.

Trade-off: the audit lane pays one extra checkout and build. Both are parallel, so this costs runner-minutes rather than wall-clock, and in exchange the e2e critical path becomes the shards themselves.

Refs #1965.

Verification

python3 -c "import yaml; yaml.safe_load(...)" on the workflow — parses; jobs resolve to changes, typecheck, test_workspaces, test_runtime_host, test_headless, test, e2e_shard, alignment_audit, e2e, storybook, with e2e.needs = [changes, e2e_shard, alignment_audit].
Confirmed the new lane's prerequisites are sufficient by running the audit locally against exactly what the job builds:

$ npm --workspace @maka/desktop run build:with-deps && node scripts/audit-alignment.mjs
alignment audit: all fixtures clean
node scripts/audit-alignment.mjs 39.478 total

This is the same prerequisite the audit had before: the e2e npm script it rode was build:with-deps && playwright test.

Grepped for other references to the audit or the job names; nothing else pins them. scripts/ci-test-plan.test.mjs asserts plan outputs, not workflow structure.
Not run locally: the workflow itself. The shard-balance claim is from CI timings on run 30798353157 and will be re-confirmed on this PR's own run.

The CDP alignment auditor rode e2e_shard 1, which made that shard ~51s longer than shard 2 for work Playwright never depended on: measured 118s/120s of balanced test time, then a 51s audit on one side only. Split it into its own job behind the same `e2e` selection gate, and add it to the aggregate `e2e` check so the required gate still covers it. The lane pays one extra checkout and build; both run in parallel, so the e2e critical path drops to the shards themselves.

Astro-Han force-pushed the ci/alignment-audit-own-lane branch from 9234d09 to 9702fb8Compare August 3, 2026 09:46

Astro-Han marked this pull request as ready for review August 3, 2026 09:58