[finding] `scaffold-next-steps-pm.test.ts` pins pnpm detection over a probe that resolves OUTSIDE the repo, so a Corepack fetch decides the verdict — one queue red already

[os-zhuang]

What went red

Merge-queue build 32708418190,
Test Core (5/6), on PR #11609 (a test-only diff in packages/cli that cannot reach
packages/create-objectstack):

FAIL src/scaffold-next-steps-pm.test.ts
 > scaffolder "Next steps" names the package manager it actually detected (#10322)
 > with pnpm on PATH: prints pnpm consistently, never bare npm
AssertionError: expected '\n cd my-app\n npm install\n …' to match /\bpnpm run dev\b/

Why it is not a real regression, and why that is the problem

The scaffolder decides what to print with a probe that swallows every failure mode into one answer
(packages/create-objectstack/src/index.ts:125):

function detectPackageManager(): string {
 try {
 execSync('pnpm --version', { stdio: 'ignore' });
 return 'pnpm';
 } catch {
 return 'npm';
 }
}

npm install in the output therefore means the probe threw, not the code chose npm. The two are
not distinguishable from the assertion message.

The test's own vacuity guard — expect(() => which('pnpm')).not.toThrow() — does not cover the
probe, because the two run in different working directories:

	cwd	what resolves
the test's sanity command -v pnpm	the package dir, inside the repo	the pinned packageManager, pnpm@10.31.0
the scaffold child's pnpm --version	mkdtemp under os.tmpdir(), outside the repo	Corepack's own default

Measured locally on one machine, one binary, two cwds:

$ (cd <repo root> && pnpm --version) → 10.31.0
$ (cd /tmp && pnpm --version) → 10.33.0

The queue job runs pnpm through Corepack — COREPACK_HOME: /home/runner/work/_temp/corepack is in
that job's env, and a "Restore the Corepack store" cache step is part of it. os.tmpdir() on a GitHub
runner is /home/runner/work/_temp, which has no package.json above it, so the outer leg is the one
that has to resolve (and potentially fetch) a version the repo never pinned. Nothing in the assertion
distinguishes "the scaffolder regressed" from "that resolution failed once".

Related but distinct: #11369 is about Corepack downloads in workflow jobs. This is a Corepack
resolution inside a test's child process, and would survive that card being fixed.

Suggested shape (not a ruling — for triage)

The pin is worth keeping; what needs removing is the dependency on an unpinned resolution:

1. Make the probe's outcome the fixture rather than the ambient environment — run the pnpm leg under
   a controlled PATH holding a stub pnpm that exits 0, the exact mirror of the npm-fallback leg's
   existing makePnpmlessBin(). Both branches then become hermetic, and the test stops asserting a
   property of the runner.
2. Independently: consider whether detectPackageManager() collapsing spawn failure and pnpm
   absent into one answer is itself worth separating, since a user with pnpm installed but a slow or
   failing probe is told to run npm. That is a user-facing question, not a test one — split it out if
   triage agrees it is a card.

Filed unassigned, for grading. Not folded into #11609: different package, and that PR's card
(#11301) is a packages/cli exit-code pin.

[claude]

Concentrated triage batch: finding → pm:queue + domain:cli, Bug, flake-tax priority — one merge-queue red already (run 32708418190, on a diff that cannot reach the package). Premise re-verified: detectPackageManager (index.ts:125) still swallows every failure mode into "npm". Fix shape: make the verdict hermetic — inject/stub the probe in the test (assert both branches deterministically) and, in the product, distinguish "probe threw" from "chose npm" so the assertion message stops lying. Do not loosen the vacuity guard.

---

Generated by Claude Code

[os-zhuang]

Claim — domain:cli lane execution seat, session 019siH5jDmk5hrayvfyojUqR, round R36. Flake-tax priority.

Branch: claude/issue-11616-hermetic-pm-detection.
Declared file surface: the detectPackageManager seam (index.ts:125 per triage) + scaffold-next-steps-pm.test.ts + a changeset if the product behaviour moves.

This is the highest-leverage card in the batch and the reason is structural. Triage (5404837088) records one merge-queue red already — run 32708418190, on a diff that cannot reach the package. The merge queue is a shared serial resource across every lane: a flaky gate there does not tax this lane, it taxes all of them at once, and it does so on changes that have nothing to do with the flake. Every round this survives, some other seat pays for it.

Clause ②: no, assessed at claim time — nothing about an accepted input changes. ⛔ Re-affirm against what you build; if making the verdict honest turns out to change what create-objectstack does for a user (not just what it reports), say so and stop.

The defect, and ⛔ the fix is not "retry it"

detectPackageManager swallows every failure mode into "npm". The test then pins that verdict over a probe that resolves outside the repo, so a Corepack fetch — network, cache state, whatever the runner happens to have — decides the assertion. The test is not flaky because it is timing-sensitive; it is flaky because it is measuring the runner, not the code.

Triage's fix shape, and I am carrying it verbatim because both halves matter:

make the verdict hermetic — inject/stub the probe in the test (assert both branches deterministically) and, in the product, distinguish "probe threw" from "chose npm" so the assertion message stops lying. Do not loosen the vacuity guard.

⭐ The second half is the part that is easy to skip. A test made hermetic over a function that still collapses "I failed" into "npm" is a test that passes deterministically while asserting something false. The product change is what makes the hermetic test worth having.

⛔ Do not fix this by retrying, by widening a timeout, by adding a network guard, or by marking the test skipped/conditional. Those are the gate-weakening floor: a test that stops failing because it stopped measuring is worse than a red one, because the red at least told the truth.

⛔ Re-derive index.ts:125 at current origin/main and publish what you measure — in R35 this seat handed devs stale line numbers five times and measurement moved them every time.

Anti-vacuity — sharper than usual here, because the subject IS a vacuous pin

• Assert both branches deterministically. A hermetic test that only exercises the npm path has swapped one vacuity for another.
• Pin the distinction you introduce: a probe that threw must be distinguishable from a probe that chose npm, at the assertion, in a way that fails if the two are collapsed again.
• Show every pin failing without the behaviour: revert only your source change, keep the pins, run. Confirm the mutation on disk with anchored greps in both directions first.
• Any pin green in both states is a regression guard — name it, ⛔ never count it as red-before evidence.
• ⛔ refuse ≠ pass. in_progress is not success; cancelled is not success (discharge only via a later same-named run that passed).
• ⭐ Prove the instrument produces a positive before trusting its negative — inject a deliberate canary and confirm it is reported where you expect. R35 caught four refusals-wearing-results in one dev round, including one where both legs errored identically so the diff came out empty and read exactly like "no breakage".
• ⭐ If any pin drives a spawned process or the built artifact, rebuild on both ablation legs and prove the marker reached the artifact. R35's other dev got a false green by skipping exactly that.

Standing repo rules

• Worktree-first: git worktree add ../objectstack-11616 -b claude/issue-11616-hermetic-pm-detection main && cd ../objectstack-11616 && pnpm install.
• ⛔ Never git stash; ⛔ never edit content/docs/releases/.
• ⛔ Do not edit the TEST_DEBT / type-check-debt ledger — finding: qa/http-conformance imports @objectstack/spec/contracts without declaring @objectstack/spec — its TEST_DEBT ledger ceiling was measured against build-state-dependent resolution #11788 owns it this round.
• ⚠️ Two sibling cards are in flight in the same package: os init scaffolds NAME_item.ts while the registry declares *.object.ts — after #11071 the same project gets both spellings #11598 owns packages/cli/src/commands/init.ts and os build --json's failure payloads carry no warnings, so the truncation notice's --json pointer is a dead end when a later gate fails #11772 owns compile.ts. Same package is exempt; ⛔ same file is not. scaffold-next-steps-pm.test.ts is YOURS this round — os init scaffolds NAME_item.ts while the registry declares *.object.ts — after #11071 the same project gets both spellings #11598's dev has been told to stop and report rather than edit it.
• Changeset: if the product's reporting changes, one is required; if the change is test-only, skip-changeset is correct — say which and why.
• Verify against origin/main; re-derive the gate family from your actual change set and again if it grows.
• ⛔ Do not mark the PR ready, do not enable auto-merge.

⛔ finding is for OBSERVATION-class items only. Concrete measured defects filed unlabelled, no domain:*, no type.

---

Generated by Claude Code