[finding] `check-cross-package-test-inputs.mjs --union-into` appends to `packages.items` but leaves `packages.count` stale, so the file it hands to the shard partitioner is self-contradictory

[os-steve]

Found while measuring the shard package set for #10016. Filed unassigned. Inert today — nothing reads the stale field — which is why this is an observation rather than a bug report.

scripts/check-cross-package-test-inputs.mjs's unionInto() reads the turbo ls --output=json payload, pushes the newly-unioned packages onto parsed.packages.items, and writes the document back:

items.push({ name, path: join(REPO_ROOT, dir) });
…
writeFileSync(listPath, JSON.stringify(parsed));

parsed.packages.count is never touched. Measured on main (c1731d023) with a changed-file list of exactly scripts/sync-template-versions.mjs:

$ node scripts/check-cross-package-test-inputs.mjs --union-into ls.json --changed changed.txt
Cross-package scans pulled into this run because the diff touched their declared inputs:
 + @objectstack/spec (declared glob matched scripts/sync-template-versions.mjs)
 + create-objectstack (declared glob matched scripts/sync-template-versions.mjs)

$ # the resulting document:
count: 0 items: ['@objectstack/spec', 'create-objectstack']

turbo ls --affected returns ZERO packages for that diff, so count stays at the pre-union 0 while items carries two — the document now says two contradictory things about its own size.

Why it is harmless today, stated so nobody has to re-derive it: the only consumer, scripts/partition-test-shards.mjs, reads parsed?.packages?.items and never count; I grepped both scripts and neither mentions packages.count.

Why it is still worth closing: partition-test-shards.mjs deliberately asserts the payload shape loudly — its header says turbo ls is experimental and "the partition script asserts its output shape loudly so an upgrade that changes it turns into a red step naming the cause, not a silently empty shard". A hand-mutated document that no longer satisfies its own schema is the input to that assertion. A future consumer (or a turbo ls upgrade that makes count authoritative) reaching for the obvious field would compute an empty or truncated shard, and the failure shape is the silent one the surrounding machinery exists to prevent: a green Test Core that tested nothing.

Two candidate fixes, both one line, and the choice is a judgment I did not want to make in passing:

• set parsed.packages.count = items.length before the write — keeps the document a valid turbo ls payload; or
• delete parsed.packages.count — makes any consumer that reaches for it fail loudly rather than read a plausible wrong number, which is more in keeping with this script's "the default for an undeclared scan is a RED GATE, never a silent skip" posture.

Either way a --self-test case pinning the invariant would keep it from rotting back.

Refs: #10016 (where this surfaced), #7802 (the failure family this script exists for).

---

Generated by Claude Code

[claude]

Claim: PM loop round 24
Session: session_01XqDQYVU5smx29ts9pAErja
Branch: claude/issue-10046-union-into-count
Worktree: objectstack-issue-10046
Domain: domain:devx
File surface: scripts/check-cross-package-test-inputs.mjs (the --union-into writer), possibly scripts/partition-test-shards.mjs (reader) (stop on breach; explain in the report)
Container & model: S, mode:subagent, model: opus (--tier on these paths returns "no path-derived mandate … This line is a FLOOR, never a clearance"; content limb no — the emitted document gains a correct count; no gate's accept/reject moves)
Clause-②: no
Serial constraints cleared: swept all open PRs for any touching cross-package-test-inputs or partition-test-shards — none. Predecessors on this file, PR #9826 (#9763) and PR #10018 (#9977), are both MERGED.

---

Generated by Claude Code

[os-steve]

{
 "issue": 10046,
 "status": "done",
 "branch": "claude/issue-10046-union-into-count",
 "pr": "https://github.com/objectstack-ai/objectstack/pull/10059",
 "premise_still_valid": true,
 "summary": "RULING 1 = MAINTAIN, on measured evidence about whose schema `count` is: `turbo ls --output=json` (turbo 2.10.10) emits it itself and keeps count === items.length in every form measured (bare 77/77, --filter 1/1, --affected 77/77). It is turbo's field, not this script's invention, so it is not ours to delete -- deleting it would make our hand-mutated document stop being a valid `turbo ls` payload, the opposite of what partition-test-shards.mjs's assert-the-upstream-shape-loudly posture exists for. Both halves ship together: the writer reconciles count inside `serializePackageList()`, the ONLY thing in unionInto() that turns the document into bytes (a `reconcile(); write();` pair would re-create the defect the first time someone added a second write path); the reader `readPackageItems()` refuses a payload whose count disagrees with items.length, naming the contradiction. RULING 2: a future disagreement is now caught in production -- the reader runs on every Test Core shard, so it is a red step naming the cause, not a self-test hope. A payload with NO count is accepted deliberately (a redundant field's absence cannot mis-shard; its disagreement can), and a present-but-wrong count is rejected, never repaired. RULING 3 verified, not assumed: old vs new partitioner over the real 77-package document, all 3 shards, produce IDENTICAL package lists, weights and bin totals (23/29/24, 781/780/780). Rulings 4 and 5 hold: no governed surface touched, no changeset, `skip-changeset` applied and read back. ONE CHANGE BEYOND THE MINIMAL DIFF, declared: `partition-test-shards.mjs --self-test` was wired to NOTHING -- not package.json, not any workflow -- so every assertion in it, including the partitioner's pre-existing determinism/coverage/balance pins, evaluated never. Adding reader-side pins to an unrun self-test would have been dead code, and the card asked for a pin that 'keeps it from rotting back', which an unrun pin cannot do. It is now a step in Lint & Repo Gates, invoked as `node ... --self-test` rather than a `pnpm check:*` alias because that alias would live in root package.json, inside the #9465 fence -- matching the release-rehearsal self-test step directly above it.",
 "tests": "All at final commit a0b947783d (worktree, turbo 2.10.10, node 22.22.2).\n\nH1 REPRODUCED then FIXED. Pre-fix on 20b9a9ce1b, changed-file list = exactly scripts/sync-template-versions.mjs: `packages.count = 0 / items.length = 2`. Post-fix: `count 2 / items 2 / AGREE true`, packageManager 'pnpm9' untouched. The old contradictory document is now refused: exit 1, `packages.count is 0 but packages.items holds 2 -- the payload contradicts itself about its own size ... Refusing to shard it.` Mutations confirmed on disk independently of the edit tool: injected tokens grepped with counts (serializePackageList 5, readPackageItems 12, 'Refusing to shard it' 1) AND removed text confirmed absent (reconcilePackageCount 0, `writeFileSync(listPath, JSON.stringify(parsed))` 0).\n\nGATE UNION re-derived with `node scripts/pm/dispatch-gates.mjs` (no paths passed; it reads the merge base itself) -- it named SIX families the dispatch brief did not, all pulled in by the lint.yml edit. Exit codes captured by redirect-then-capture, never through a pipe; verdict lines quoted from the gates' own output: check:cross-package-test-inputs 0 'OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.' | check:node-version 0 'OK (30 setup-node step(s) across 27 workflow(s), all on Node 22).' | check:required-contexts 0 | check:shard-attestation 0 | check:type-check-coverage 0 'OK -- 64/77 workspace packages type-checked (plus the root), 13 in the DEBT ledger' | check:workflow-status-functions 0 'OK (scanned 27 workflow file(s), 46 job(s), 24 job-level if: expression(s))' | check:nul-bytes 0 'OK (scanned 6358 text file(s) ... 0 untracked-not-ignored)' | partition-test-shards.mjs --self-test 0 'partition-test-shards: self-test OK' | check-cross-package-test-inputs.mjs --self-test 0 'All 57 self-test cases passed.'\n\nDECLARED NARROWING (one): check:type-check-debt exits 1 here, refusing to --re-measure because 55 workspace dependencies have no built dist/*.d.ts. NOT caused by this branch -- the identical refusal reproduces on the unmodified shared origin/main checkout with none of this diff present, and the diff touches ZERO TypeScript files (3 files: 2 .mjs, 1 workflow YAML). The half of that gate that reads lint.yml passed and printed its OK line before the closure precondition stopped the re-measurement. Building the 77-package closure would hold the container's shared heavy-verify lock ~10 min to re-measure a TS debt ledger this change cannot move; CI builds the closure before that step.\n\nABLATION -- REBUILD STATEMENT: no rebuild leg applies, and that is checked rather than assumed. Both scripts import only `node:` builtins, are executed directly as source by `node scripts/*.mjs`, appear in no vitest/test-source-alias ledger, and are reached through no package build -- there is no dist/ between the edit and the measurement, so the dist preflight is inapplicable rather than skipped. Both legs run from the COMMITTED state and restored with `git checkout HEAD -- <path>`; tree AND index confirmed clean afterwards via `git status --porcelain` (not `git diff HEAD`). Direction predicted before each run. WRITER leg: predicted red on exactly 2 of the 5 new cases (the other 3 already have a correct count, so a missing assignment is invisible to them) -- observed exactly that, '2/57 self-test case(s) failed', naming 'count follows an appended item' and 'count follows an empty list down'; end to end the document regressed to count 0 / items 2 and the READER refused it (exit 1, 0 packages emitted). READER leg: predicted red at the FIRST must-throw case since selfTest() throws rather than collecting -- observed 'Error: payload: count 0 beside 2 items was accepted', and the contradictory document was silently accepted again (exit 0, @objectstack/spec selected), reproducing the pre-fix behaviour on demand.\n\nRULING 3 PROOF: old (origin/main, hash-verified 7619ffcc37b9c3a6dedd5d4d503be88f9e5e432c) vs new partitioner over the same real 77-package document, --exclude @objectstack/dogfood: shard 1 23/76 weight 781, shard 2 29/76 weight 780, shard 3 24/76 weight 780, all bins 781/780/780 -- package lists IDENTICAL on all three, 76 packages placed. Unioned-document path unchanged too: 'shard 1/3: 1/2 packages, weight 414', @objectstack/spec, before and after.\n\nH2 REPO-WIDE READ CHECK -- nothing reads `count`, confirmed beyond the two scripts: zero matches repo-wide for packages.count / packages?.count / bracket forms (excluding node_modules, dist, .git, .turbo, coverage); zero `jq` steps touching count in any workflow; zero committed *.json fixtures containing a \"count\" key at all -- so removal would NOT have been a test change, but that is moot given the schema evidence. The document's entire lifecycle is ci.yml lines 342-385: written by turbo ls to $RUNNER_TEMP/turbo-ls.json, mutated by --union-into, read by partition-test-shards.mjs. Never jq'd, never uploaded as an artifact, never read by anything else. No `packageCount` anywhere.\n\nH3 VERDICT -- the emptiness guard does NOT distinguish the two cases, and structurally cannot at that layer; the shape assertion does fire for its stated job. Measured: items key removed -> exit 1; items an object -> exit 1; EMPTY (count 0, items []) -> exit 0, 'shard 1/3: 0/0 packages', 0 stdout lines; NON-EMPTY (count 2) -> exit 0, 1 line; SELF-CONTRADICTORY (count 0, items 2) -> exit 0, silently accepted (pre-fix). Zero stdout then hits ci.yml's `No packages on this shard -- nothing to test.` short-circuit: a green Test Core that ran nothing. But zero is FREQUENTLY the correct answer here, unlike #4690's scanner -- a docs/scripts-only PR genuinely affects no package, and with 3 shards any run with fewer than 3 packages legitimately empties one -- so a blanket red-on-empty at the partitioner would be wrong. Real mitigations already exist: every FAILING producer falls back to the FULL package list (three branches in ci.yml), and steps run under `bash -e`, so a crashing selection cannot yield an empty shard. The residual hole is a producer exiting 0 with a wrong empty answer, and the one decidable signal for it is unchecked: ci.yml checks the exit status of `git diff --name-only \"$SCM_BASE\" HEAD` but never its EMPTINESS, and an empty changed-file list on a pull_request event is not a legitimate state. Filed as #10057 rather than fixed, per the card. Note the shard attestation (#6082) does not cover it -- it attests 'shard N ran and every step passed', which is exactly what a shard that tested nothing does.\n\nH4 FULL FIELD AUDIT -- the document has exactly five fields, verified exhaustively (Object.keys at root and packages level; all 77 items share ONE key set, name+path). root = {packageManager, packages}; packages = {count, items}; item = {name, path}. No timestamp, no filter echo, no packageCount. Does the union step keep each true? packageManager 'pnpm9' -> YES, a workspace-level fact unaffected by appending items (now pinned by a self-test case). packages.items -> YES, the append is the intended mutation. items[].name -> YES, same convention. packages.count -> WAS NO, this card. items[].path -> NO, a SECOND field left untrue: turbo emits repo-RELATIVE paths (0 of 77 absolute) while unionInto pushes join(REPO_ROOT, dir), i.e. absolute. Inert today (measured: weight 414 either way from the repo root) and deliberately NOT fixed here -- path feeds countTestFiles() feeds the weight feeds shard placement, so touching it could move packages between shards, which ruling 3 fences; the correct normalisation is also a genuine judgment call (turbo-faithful relative is cwd-fragile -- measured weight 0 from another cwd, silently absorbed by the LPT partitioner -- while all-absolute stops being a turbo payload). Filed as #10056.",
 "open_questions": [],
 "out_of_scope_findings": [
 "filed as #10056: `--union-into` pushes an ABSOLUTE packages.items[].path while turbo ls emits repo-relative ones (0 of 77 turbo entries absolute), so the document is heterogeneous; inert today but path feeds the weight feeds shard placement, so fixing it could move packages between shards -- and the correct normalisation is a judgment call, not mechanical",
 "filed as #10057: nothing distinguishes 'zero packages affected' from 'the affected-set computation silently produced nothing' -- ci.yml checks the exit status of `git diff --name-only \"$SCM_BASE\" HEAD` but never its emptiness, and an empty changed-file list on a pull_request event is a decidable 'selection failed' signal"
 ]
}

---

Generated by Claude Code