Skip to content

test(runtime): withhold and assert the expected refused a read on noise in 16 passing pins - #10985

Merged
os-elon merged 1 commit into
mainfrom
claude/issue-10629-runtime-test-noise-remainder
Aug 22, 2026
Merged

test(runtime): withhold and assert the expected refused a read on noise in 16 passing pins#10985
os-elon merged 1 commit into
mainfrom
claude/issue-10629-runtime-test-noise-remainder

Conversation

@os-elon

Copy link
Copy Markdown
Collaborator

Fixes#10629

Sixteen passing @objectstack/runtime fixtures printed expected read failures into the shared shard log. They now withhold and assert that noise instead of emitting it — the shape PR #10630 ruled for this class, applied verbatim.

The re-measured population — and where the card's table was short

⛔ The card's counts were taken pre-#10630, so the first act was to re-run the enumeration at this branch's merge base (926778bce0), not to work to the recorded number.

pnpm --filter @objectstack/runtime exec vitest run --reporter=default --maxWorkers=2
Test Files 179 passed (179) Tests 2679 passed (2679)
refused a read on: 134
card (pre-#10630)re-measured at 926778bce0
refused a read on lines134134 — unchanged
files1516
top emitternotifications.hono.integration.test.ts (52)same, 52

The total did not move, and neither did any row — but the card's table lists 15 files summing to 133, one short of the 134 it states. The missing row is default-datasource-plugin.test.ts at 1 line, dropped from the table but present in the total. Nothing changed under #10630 (that PR's two files were already excluded from this 134 by construction); the card's table was simply a row short of its own total. Worked to 16, not 15.

The card counted one channel. Attribution by the reporter's own stderr | file header on a --no-file-parallelism run shows the driver refusal and the engine frame are paired 1:1, so the real population is 268 lines:

lines (each channel)file (under packages/runtime/src/)probe that emits it
52notifications.hono.integration.test.tsresolveUserAuthzGrants — six sys_*tryFinds
19package-list-commits-org-scope.integration.test.tsrunBuildProbes view read on anything
11package-revert-commit-org-scope.integration.test.tsrunBuildProbes view read on anything
7cascade-delete-multivalue-lookup-real-driver.integration.test.tsprobeInstallOrganizations
7seed-multi-value-lookup-real-driver.integration.test.tsprobeInstallOrganizations + resolveSoleOrganizationId
6bulk-write-real-driver.integration.test.tsprobeInstallOrganizations + resolveSoleOrganizationId
5sandbox/undeclared-field-write-driver-split.integration.test.tsprobeInstallOrganizations
5federated-boot-binding.test.tsboot metadata load on sys_metadata
5expand-nested-fields-join-key.integration.test.tsprobeInstallOrganizations
4package-revert-commit-attribution-org-scope.integration.test.tsrunBuildProbes view read on anything
3batch-row-http-status-real-driver.integration.test.tsprobeInstallOrganizations
3batch-row-driver-text-real-driver.integration.test.tsprobeInstallOrganizations
3preserve-audit-real-driver.integration.test.tsprobeInstallOrganizations
2seed-loader-driver-text-real-driver.integration.test.tsprobeInstallOrganizations + resolveSoleOrganizationId
1default-datasource-plugin.test.tsthe row the card's table omitsprobeInstallOrganizations
1sandbox/nested-write-real-sqlite.integration.test.tsprobeInstallOrganizations
13416 files+ 134 matching engine frames = 268 lines

Before / after

Same command, same reporter, at ef25e4a341:

before (926778bce0)after (ef25e4a341)
Test Files179 passed (179)179 passed (179)
Tests2679 passed (2679)2679 passed (2679)
refused a read on1340
ERROR Find operation failed1406

Per file, all sixteen go 134 → 0 and 134 → 0, measured on a --no-file-parallelism run over exactly the 16: Test Files 16 passed (16) · Tests 78 passed (78), noise count 0 — against 16 / 78 / 268 on the same command at the merge base. The test count is identical on both sides: nothing skipped, quarantined, .only-scoped, retry-wrapped or deleted.

The 6 surviving ERROR Find operation failed frames are not residue of this population. All 6 come from cross-field-refusal-operand-withhold.test.ts and carry the #7929 cross-field refusal — a different feature that shares the engine's frame, with no table to key on and no driver refusal beneath it. Out of this card's scope; filed as #10983 (finding · domain:cli, no pm:queue).

The #10630 pattern, and the one thing done differently

#10630's landed diff is the ruled shape and was read before anything was written. Preserved verbatim:

  • two sinks — the driver's logger (assigned; a protected field with a console default, the idiom its own doc comment names) and the engine's logger behind a Proxy on error alone, so every other logger method resolves to the engine's own;
  • the predicate — a line is withheld only when it names an expected table and carries that same table's no such table reason. Verified across all 134: every refusal names its own missing table as the reason, 0 mismatched;
  • the engine gate is the narrower of the two — a frame is withheld only when it sits directly above a driver refusal the capture already recognised (the pending-refusal counter). A DATABASE_ERROR on the same table from any other cause is not recognised by the driver sink, so its frame reaches the log with both halves intact;
  • the driver sink mirrors { warn, error } so SqlDriver.logDurabilityFailure still finds an error channel and does not silently re-level durability messages this card is not talking about;
  • count what was withheld, and assert the counts.

Differently: it lives in one module rather than sixteen copiespackages/runtime/src/expected-read-refusal-noise.ts, test-only (nothing in src/index.ts imports it, and tsup's single entry is src/index.ts, so it is not bundled; it imports no vitest, so the assertions stay visible in the fixture that owns them). #10630 wrote the predicate inline because it had two call sites; this has sixteen, and sixteen copies of one predicate is sixteen places for it to drift — including drifting looser, the direction that turns a pin back into a mute with nothing going red. The mechanism is unchanged; only the duplication is gone.

The assertion surface is expect(capture.silentChannels()).toEqual([]) — one call that still makes a silent channel name itself in the diff, which is what #10630's "one assertion per channel" bought at sixteen times the bulk.

Per-file judgement: expected vs diagnostic

The fence that mattered most. Nothing was quieted for looking like noise. Each line was traced to its emitting call site through the engine frame's own stack, and every one is a fail-soft probe — a read the runtime issues to learn whether something is installed, whose missing-table answer it swallows by design, with the swallow documented at the source:

probethe swallow, in its own words
core/src/security/resolve-authz-context.tstryFind on six sys_*the resolver is fail-closed and must always resolve
objectql/src/engine.tsprobeInstallOrganizationscatches isMissingTableErrorand only that, which its own doc comment calls "the one benign cause"; anything else propagates
metadata-protocol/src/seed-loader.tsresolveSoleOrganizationId"sys_organization may not exist (single-tenant runtime) — ignore"
objectql lifecycle governance snapshot"No sys_organization (single-tenant kernel) — tenant overrides n/a"
metadata-protocol/src/build-probes.tsrunBuildProbesturns the failure into a view_read_failed publish issue, not an exception; the suites assert res.success and never read that issue
boot metadata load on sys_metadatasurvives the miss; the fixture creates no metadata store

No file turned out to be a real failure signal, so no stop-and-report was needed. The one that came closest is federated-boot-binding.test.ts, whose 5 lines appear only in the OS_SKIP_SCHEMA_SYNC case — the flag means DDL is managed out of band, and this fixture manages none, so an absent sys_metadata is what the flag produces rather than a defect. The capture is scoped to that single case on purpose: the ordinary boot beside it creates the table and stays fully loud if it ever stops.

Two other scoping decisions in the same spirit:

  • sandbox/undeclared-field-write-driver-split.integration.test.ts has a SQL family and a schemaless family. The memory boot declares an empty expectation rather than skipping the assertion, so the shared afterEach still fails loudly if a boot ever forgets to install a capture.
  • notifications.hono.integration.test.ts withholds all six sys_* tables but requires only the five read on every grant resolution. sys_setting is read on some routes only; requiring it would turn a single-test -t run red without meaning anything. This is test(runtime): withhold and assert the expected error noise in two passing pins #10630's own distinction, kept.

Assertions sit in afterEach where every test in the file provokes the probe, and in the test bodies where only some do (batch-row-http-status — its not-found case never writes, so never resolves a system-write org; notifications.hono — discovery and the anonymous 401 resolve no grants; default-datasource-plugin — one of twelve cases writes through the booted engine). An afterEach assertion in those files would redden a test for a reason that has nothing to do with it.

Vacuity check — every new assertion can fail

⛔ An assertion that expected noise was emitted is worthless if it cannot go red. Proven by ablating the shared predicate so nothing is ever withheld (the module is a relative in-package import that vitest transforms from source — no dist is involved and no rebuild is required, unlike a dist-resolved ablation), then re-running the same 16 files:

Test Files 16 failed (16)
Tests 58 failed | 20 passed (78)
noise back in the log: 268

All sixteen files go red, and every silent channel names itself:

+ "the driver's read refusal for 'sys_organization' was never emitted",
+ "the engine's 'Find operation failed' frame for 'sys_organization' was never emitted",

The noise count returning to exactly 268 is the second half of the proof: the withhold is precisely what removed those lines, and nothing else was suppressing them. The ablation was then reverted and the file proven byte-identical to the committed blob (git hash-object against git rev-parse HEAD:...).

Gate verdicts

Union derived at the final commit with node scripts/pm/dispatch-gates.mjs (no paths passed — the script takes the change set from the merge base itself), and run there. ef25e4a341, exit codes captured before any pipe.

gateits own verdict line
check:type-check-debt--re-measure: OK — 33 ledger entr(ies) re-measured in 191.2s, 1908 raw tsc error(s) total, none above its recorded number.
check:type-check-coverageOK — 64/77 workspace packages type-checked (plus the root), 13 in the DEBT ledger
check:engine-double-contractOK self-test: separates engine doubles from driver doubles AND from scoped repositories on BOTH write verbs
check:where-matcherOK self-test: separates conjoining, early-returning, combinator-blind and refusing
check:query-options-erasure✓ ratchet holds: 67 unswept non-test site(s) in 17 file(s), none new
check:slot-lookup✓ ratchet holds: 107 unswept site(s) in 25 file(s), none new … baseline key set verified against 926778b: no files added.
check:cross-package-test-inputsOK: 13 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
check:test-source-aliascheck-test-source-alias --self-test OK
check:type-source-resolutioncheck-type-source-resolution --self-test OK
check:changeset-gate-self-tests✓ 118 + 212 + 116 assertions over real temp git repos
check:objectui-changesetexit 0, self-test assertions pass
check-empty-changeset✓ No empty-frontmatter changeset introduced by this diff (1 declaring changeset(s) added).
check-changeset-no-major✓ This diff introduces no major bump.
check-adr-0087-registration✓ this PR adds no declared-breaking changeset (1 non-breaking changeset(s) seen).
check-plugin-teardown-shape✓ 57 Plugin implementation(s) across 4416 source(s) … every teardown-shaped method sits beside a real destroy()
check-ci-filter-parityOK: all 82 declared cross-package glob(s) (71 unique) are covered
check-nul-bytesOK (scanned 6324 text file(s) … no raw ASCII control bytes).

No gate weakened, no threshold moved, no baseline raised. The @objectstack/runtime TEST_DEBT entry sits at its recorded 227 — measured directly with the test exclusion lifted, before and after. One error this branch briefly introduced (an unused type import) was repaired at the source rather than absorbed by the ledger's headroom.

Changeset

.changeset/runtime-expected-read-refusal-noise.md, @objectstack/runtime: patch. ⚠️ A tests-only PR would normally take the skip-changeset label here, but this one adds a real module under packages/runtime/src/, and check-empty-changeset rejects an empty-frontmatter changeset outright — so a declaring one is the only shape that satisfies both. It states plainly that no runtime behaviour changes; the published dist is unaffected, since tsup's only entry is src/index.ts and nothing there imports the new module.

Out-of-scope findings

Both filed unassigned, finding + domain:cli, no pm:queue — grading is triage's. Labels verified by read-back on both.


Generated by Claude Code

…oise in 16 passing pins (#10629)
Sixteen passing `@objectstack/runtime` fixtures printed expected read
failures into the shared shard log: 134 `[sql-driver] DATABASE_ERROR — the
backend refused a read on '<table>'` envelopes plus their 134 matching
`ERROR Find operation failed` engine frames, 268 lines in all. Turbo
interleaves package logs without attribution, so an ERROR-shaped line from a
green test is indistinguishable from a real failure — lines of this shape
were once lifted verbatim into a p1 flake signature (#10293).
Every one is a fail-soft probe whose missing-table answer the runtime
swallows by design: `resolveUserAuthzGrants`' six `sys_*` `tryFind`s,
`ObjectQL.probeInstallOrganizations`,
`SeedLoaderService.resolveSoleOrganizationId`, the lifecycle governance
snapshot, `runBuildProbes`' view read, and the boot metadata load. Each was
judged expected rather than diagnostic; none was silenced on the strength of
looking like noise.
Applies PR #10630's ruled shape verbatim, factored into one shared test-only
module: withhold only a line naming an expected table AND carrying that same
table's `no such table` reason, count what was withheld, and assert the
counts. The engine frame is withheld only when it sits directly above a
driver refusal the capture already recognised, so an identically-shaped
fault from another cause still reaches the log with both halves intact.
No test skipped, quarantined, retried or removed: 179 files / 2679 tests
pass before and after.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019bmVFqoQPq63zhKrxdYG1r
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

11 anchor(s) derived from 1 changed package(s); no hand-written page names any of them. ✅

What this run could not see
  • 2 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 45 of 221 client-bound route-ledger rows — the other 176 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run: node scripts/docs-audit/affected-docs.mjs --bridge-coverage

Coarse fallback — 23 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 471cfbd97501f5974a5437ef24d2ca266e6e8edepackageMentionDocs.

Which tree this was computed on

This run read content/docs from d0635b8585236e27252f92fa3e52dfe816b2bfda — the merge of head ef25e4a34148e7c0d77fc343cb4ccae90847cd54 into base 471cfbd97501f5974a5437ef24d2ca266e6e8ede, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin d0635b8585236e27252f92fa3e52dfe816b2bfda && git checkout d0635b8585236e27252f92fa3e52dfe816b2bfda
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 471cfbd97501f5974a5437ef24d2ca266e6e8ede ef25e4a34148e7c0d77fc343cb4ccae90847cd54 && git checkout -B drift-repro 471cfbd97501f5974a5437ef24d2ca266e6e8ede && git merge --no-ff ef25e4a34148e7c0d77fc343cb4ccae90847cd54
node scripts/docs-audit/affected-docs.mjs --json 471cfbd97501f5974a5437ef24d2ca266e6e8ede

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 32546382828 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Console Pin Gate — 失败步骤: Build the Console SPA at the pinned objectui SHA

    ✗ Build failed in 6.01s
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 66 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Merged via the queue into main with commit 13a6cb4Aug 22, 2026
32 checks passed
@os-elon
os-elon deleted the claude/issue-10629-runtime-test-noise-remainder branch August 22, 2026 02:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/lteststooling

Projects

None yet

2 participants

@os-elon@claude