TV6 fails under Node on `main`, and has since the merge of #326

[taras]

What is failing

TV6: default and named mappings resolve; the agent prop registers additional agents
(packages/test-agent/tests/components.test.ts:316) fails under Node only.
Deno and Bun pass the same tier in the same runs.

expect(received).toEqual(expected) // deep equality

 Array [
- "pass",
+ "fail",
 ]
 at packages/test-agent/tests/components.test.ts:316:54

The assertion reads run.results.map((entry) => entry.status), so the failure
names the tier and nothing about which of TV6's three mappings — default,
session="named", or agent="extra" — did not resolve.

Where

main is red, and has been since 2026-08-04 19:49 UTC:

run	commit	conclusion
30963870840	209bf218 (merge of #329)	test-node failed on TV6
30945023598	511776ef (merge of #326)	test-node failed on TV6
30940821364	7d7bdf2f	success

Two consecutive failures on two different commits, on the same test, in the same
job. The last green main is 7d7bdf2f, so the first red is the merge of #326.

What complicates that reading

#326's own pull request runs were green — 30944076192
(30b93d1f) and 30942960538
(b31f3647) — and a pull_request run checks out refs/pull/N/merge, so a
merge against main as it then stood was tested and passed. Whatever the cause,
it separates the push run from the pull_request run of nearly the same tree.

The failure has not been reproduced locally; both observations are from CI.

Why this was not noticed

main carries no required status checks and nothing announces a failed run
there. #331 is the fix for that; this issue is the failure it would have
reported.

[taras]

TV6 passes on a pull request run of the same tree

#333's CI ran on refs/pull/333/merge — 209bf218 plus changes confined to .github/workflows/, AGENTS.md, specs/decisions.md, and a new scripts/tests/ file. Nothing under packages/test-agent/.

test-node passed, TV6 included.

So the same test-agent code that failed on main twice in a row passes on a pull request run. That rules out the reading the two consecutive failures invited — a deterministic regression from the merge of #326 — and leaves two candidates:

1. TV6 is flaky under Node, and the two main failures were an unlucky pair rather than a state change. Its three scenarios (default, session="named", agent="extra") each spawn a relaunched worker, and the failing run spent 20.4s in the test.
2. Something differs between the push and pull_request runs of nearly the same tree — runner contention, caching, or event-dependent behaviour.

The assertion still reports only ["fail"] versus ["pass"] (components.test.ts:316), so neither candidate can be told apart from CI output as it stands. Whatever the cause, that is worth fixing first: the failure names the tier and nothing about which of the three mappings did not resolve.

Frequency is now the thing to measure, and #333 is what will measure it — every red main from here on reports itself.

[taras]

Root cause found, reproduced locally, and fixed in #334.

The failure is the <Test> timebox, not a mapping defect. Every <Test> body runs under the fixed 20-second timeout (TEST_TIMEOUT_MS, packages/testing/src/components.ts; specs/testing-spec.md "Each test has a fixed 20-second timeout"). Under Node, each <Prompt> that opens a session spawns a fresh tsx --tsconfig tsconfig.node.json packages/cli/src/node.ts worker — ~6.5s per spawn on a loaded CI runner (Deno spawns the same worker in ~2s, which is why test-deno runs TV6 in 6s). TV6 was the only test with three spawns inside one <Test>, so its body sat exactly at the deadline:

run	TV6 duration	outcome
30940821364 (last green main)	20,012 ms	pass
30945023598 (merge of #326)	20,395 ms	fail
30963870840 (merge of #329)	20,379 ms	fail
30948136496 (unrelated PR branch)	20,452 ms	fail

When the box fires, the result is kind: "timeout" — "test timed out after 20 seconds" — but the suite asserts statuses only, so CI showed a bare fail. Reproduced deterministically on a Mac by saturating the CPU while running TV6 under Node: same ~20.5s failure, and the captured run.results show the timeout kind.

Why the merge of #326 looked causal: it wasn't. #326 added tests only, which shifted parallel load under node --test just enough to move TV6 across a line it already straddled — and the same failure hit an unrelated PR branch (run 30948136496) in the same window. The green pull_request runs of #326 were the same coin flip landing the other way at 20.0s.

Fix (#334): one <Test> per mapping — one spawn per test instead of three, and a future failure names which mapping broke (the observability gap this issue calls out).

Open product question, deliberately not decided in #334: the fixed 20s test cap sits below the 120s contextual prompt timeout (Config.timeout, packages/runtime/config.ts), so a prompt the runtime considers healthy can never finish inside a test. Worth deciding whether the <Test> timeout should be configurable or track the contextual timeout — happy to file that separately if you want it.