Skip to content

ci: give the turbo test task something that can genuinely fail - #112

Merged
os-elon merged 3 commits into
mainfrom
claude/issue-72-ci-test-step
Aug 18, 2026
Merged

ci: give the turbo test task something that can genuinely fail#112
os-elon merged 3 commits into
mainfrom
claude/issue-72-ci-test-step

Conversation

@os-elon

Copy link
Copy Markdown
Collaborator

Fixes#72

The problem, reproduced on this branch's base

pnpm turbo run test — the last of the three steps in the required build job — executed zero tasks. No package in the workspace declared a test script, so the step passed by having nothing to do:

 • Packages in scope: @objectos/docs
• Running test in 1 packages
WARNING No tasks were executed as part of this run.
Tasks: 0 successful, 0 total

What this does

Adds @objectos/ci-scripts (tools/ci-scripts/), a workspace package whose test script runs the self-tests the repository's CI scripts declare. Today that is one: check-translation-output.mjs --self-test, which drives 20 fixtures and asserts both that every case fires exactly the rules it declares and that every rule the script enforces has a fixture able to make it fail.

pnpm turbo run test now reports 1 task, and the step exits 1 when the self-test does.

ci.yml is not touched. The step was already correct; it had nothing to find.

Where the runner lives, and why

The scripts stay at .github/scripts/ — the workflows invoke them by that path. Turbo only discovers tasks in workspace packages, so the runner needs a package. A test script on apps/docs would claim to test the docs site, which is the same species of misleading signal this card is about, so the runner gets a package of its own and pnpm-workspace.yaml grows a tools/* entry. The only lockfile change is the new importer (2 lines).

Turbo caching

Turbo hashes a package's own directory by default, so an edit under .github/scripts/ would have replayed a cached green. The test task now names the dependency explicitly, following the pattern turbo.json already uses for content/docs/**:

"inputs": [
"$TURBO_DEFAULT$",
"$TURBO_ROOT$/.github/scripts/**",
"$TURBO_ROOT$/apps/docs/lib/i18n.ts"
]

i18n.ts is in there because the self-test reads it for the locale list. Verified by hash rather than by reading the config: editing the script moved @objectos/ci-scripts#test from cc46e783b02e6cfc to 9cde36a4a663c5bc, and restoring it brought the hash back.

Keeping the step from going inert again

Two guards in the runner, both loud rather than skipping:

  • the list of self-tested scripts being empty is a failure, so removing the last entry cannot quietly restore the zero-task pass;
  • a script under .github/scripts/ that dispatches a self-test flag but is not listed is a failure naming the file, so a future self-test joins the CI step by being written rather than by someone remembering the list.

The second guard is best-effort by construction and says so in the source: it matches the quoted flag literal, not the words in a comment, and a script spelling its flag some third way goes unnoticed. It narrows the gap; the empty-list guard is the one that closes it.

Proof it can go red

Not inferred — reproduced. Weakening a rule the step covers (making checkUnsafe return no findings):

✗ script tag in the translation fired [—] expected [unsafe]
✗ javascript: URL in the translation fired [—] expected [unsafe]
✗ on*= handler in the translation fired [—] expected [unsafe]
✗ script tag in the ENGLISH source fired [—] expected [unsafe]
✗ self-test: 4 case(s) did not behave as declared
✗ 1 of 1 self-test(s) failed
ERROR run failed: command exited (1) # pnpm turbo run test → exit 1

Restoring the file: Tasks: 1 successful, 1 total, exit 0.

Both inertness guards were reproduced the same way — emptying the list fails; a fixture script under .github/scripts/ that dispatches the flag without being listed fails and names itself; a script that only mentions the flag in a comment does not fire it.

What this does not claim

The self-test is not newly executed in CI. translations.yml has run it since it landed, and its path filter includes .github/scripts/check-translation*.mjs. What changes is where it runs:

  • Translations is not a required check — build is the only one — so a red self-test there does not block a merge today;
  • translations.yml has no merge_group: trigger, so nothing runs it inside the merge queue. ci.yml does.

So the honest description of the gain is enforcement and merge-queue coverage, not new coverage. The duplication is deliberate and costs about half a second; whether translations.yml should keep its own copy of the step is a maintainer call and is not made here.

Deliberately not run by this step: check-translations.mjs and check-translation-ownership.mjs. Neither declares a self-test mode, and running them for real would make the required build job fail on the corpus's translation debt — which is the Translations workflow's job, and reported-not-blocking there by design.

Gates

Clean install, then the build job's steps in order, all on 34a8f00:

stepexit
pnpm install --frozen-lockfile0
pnpm turbo run type-check --continue0 (1 task)
pnpm turbo run build0 (1 task)
pnpm turbo run test0 (1 task, was 0)

Translation checks unaffected: check-translations.mjs exit 0, check-translation-ownership.mjs with a changed-files list exit 0, check-translation-output.mjs --report exit 0.


Generated by Claude Code

`pnpm turbo run test` — the last step of the required `build` job — executed
zero tasks, because no package in the workspace declared a `test` script. A
green step that cannot go red reads in CI exactly like one that is protecting
something.
Add `@objectos/ci-scripts`, a workspace package whose `test` script runs the
self-tests the repository's CI scripts declare — today that is
`check-translation-output.mjs --self-test`, 20 fixtures asserting every rule it
enforces has a case able to make it fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Yaqu7kmKZM3tRPd9Y4xivo
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The CI test job executes nothing and cannot go red

2 participants

@os-elon@claude