Skip to content

fix(metadata-protocol): propagate the metadata-store 503 out of getMetaDiagnostics (#8855) - #8923

Merged
hotlong merged 2 commits into
mainfrom
claude/issue-8855-diagnostics-store-outage-503
Aug 15, 2026
Merged

fix(metadata-protocol): propagate the metadata-store 503 out of getMetaDiagnostics (#8855)#8923
hotlong merged 2 commits into
mainfrom
claude/issue-8855-diagnostics-store-outage-503

Conversation

@hotlong

Copy link
Copy Markdown
Contributor

Fixes#8855

Implements triage ruling 5302846257 as written: classify by error type through the existing discrimination and propagate the 503. Not "delete the catch"; no payload field added.

The defect

getMetaDiagnostics sweeps every metadata type and publishes four numeric facts about the corpus. Its per-type read was wrapped in an untypedcatch that continued. The comment above it named a benign reason — "type not listable in this kernel scope" — and that reason is genuinely real, which is why this PR narrows the catch instead of removing it. The catch took everything else with it, including the one error the callee exists to raise.

getMetaItems classifies a failed sys_metadata read by error type through rethrowUnlessMetadataStoreUnprovisioned: it returns normally for the one benign driver reason (isMissingTableError — the table is not provisioned yet, so items: [] IS the truth) and throws a 503 for every other read failure. That is the discrimination #5532 introduced so an outage would stop looking like emptiness. getMetaDiagnostics caught the 503 back into emptiness one layer up, then published the emptiness as a number.

Measured before writing anything

The card's repro was written but never run, and triage asked for prediction-then-measurement. Prediction was recorded first; the run matched it exactly. Protocol-level, on origin/main @ 8664a2c99, with an engine whose every read rejects:

[outage: connect ECONNREFUSED 10.0.0.5:5432] RESOLVED
total=0 scannedTypes=26 scannedItems=0 Object.keys(stats).length=0
[benign: SQLITE_ERROR: no such table: sys_metadata] RESOLVED
total=0 scannedTypes=26 scannedItems=0 Object.keys(stats).length=26
every stat = {"count":0,"locked":0,"packages":[]}

The benign run is what makes the harm legible — it is the same payload minus the stats. Two user-visible harms out of one catch:

  • stats[t] is never written, so an unreadable type is absent from the response rather than zero. The Studio directory tile the field's own doc names loses the type, byte-shaped like an environment that declares none of it.
  • total counts entries that failed validation, and a store nobody can read contributes none — so the endpoint whose whole job is reporting problems answered total: 0 at the exact moment it could read nothing. Green was the failure mode.

scannedTypes reported the full 26 in both runs: it is computed from the intent (targetTypes.length, fixed before the loop) and never decremented on continue.

The fix

One narrowed catch inside one method. A 503 arriving from the read is rethrown unchanged; every other failure still skips that one type, so a kernel scope that cannot enumerate one type does not fail the whole governance sweep. ADR-0110 D3: a miss and an outage are different facts with opposite dispositions, and a consumer must never read one as the other.

The envelope is propagated, not rebuilt. Re-running isMissingTableError / rethrowUnlessMetadataStoreUnprovisioned on what arrives here would re-wrap an already-shaped 503 in a second one and displace the driver error riding as cause — the object logWithheldServerFault prints for the operator (#5437). isMissingTableError walks the cause chain, so that re-wrap is not hypothetical. The pin asserts cause by identity, which is what proves propagation rather than reclassification.

The test is status === 503 rather than the narrower code check deliberately, in the direction rethrowUnlessMetadataStoreUnprovisioned's own doc argues: a false "benign" silently mis-answers a question nobody can re-ask, while a false "real" costs one 503 the caller can retry.

Scope held

Verification

New pin packages/metadata-protocol/src/protocol.diagnostics-store-outage.test.ts carries the discriminating control in the same file, because a refusal pin with no positive control cannot show the discrimination is selective rather than blanket:

caseexpected
outage, whole-corpus sweep503 envelope, cause identity
outage, single-type sweep503
outage vs first boot, side by sidebenign answers, outage throws
unprovisioned sys_metadataresolves, every type present at count: 0
a type genuinely not listableresolves, that one type skipped, the rest counted
healthy storeresolves, real counts

"0 problems" is the right answer in the benign cell, and it is exactly the answer a blanket change would have kept producing in the wrong one.

Ablation — direction predicted before running: ordinary red, on the outage half only. Restoring the bare } catch { continue; } gives 3 failed / 3 passed, and the three reds fail by resolving with the defect's payload verbatim:

AssertionError: expected a rejection, but the sweep resolved with
{"entries":[],"total":0,"scannedTypes":26,"scannedItems":0,"stats":{}}

The three benign/selective/healthy controls stay green through the ablation — that separation is what shows the change is the outage split and not a blanket throw.

Suites (all at the head quoted below):

  • @objectstack/metadata-protocol — 103 files / 1493 tests passed
  • Downstream consumers (prefix filter, ...@objectstack/metadata-protocol): the three that touch this surface — @objectstack/objectql 210 files / 3675 tests, @objectstack/rest 118 files / 1948 tests, @objectstack/client 23 files / 301 tests. objectql carries diagnostics-clean-baseline.test.ts and is the consumer that went red on fix(metadata-protocol): fold publish Phase 2's object-table lookup at the producer #8867 after a green edited-package suite.

Gates — derived from the actual changed paths via scripts/pm/dispatch-gates.mjs, then re-run at the post-merge head: check:changeset-gate-self-tests, check:cross-package-test-inputs, check:durability-log-level, check:filter-alias-parity, check:objectui-changeset, check-adr-0087-registration, check-changeset-no-major, check-empty-changeset, plus the convention-triggered check:query-options-erasure, check:type-check-coverage and check:type-check-debt --re-measure, and check:nul-bytes / check:engine-double-contract / check:error-code-casing / check:adr-anchors added beyond the derived list. All pass.

TEST_DEBT measured directly on a built workspace, since @objectstack/metadata-protocol sits in the DEBT ledger and pnpm typecheck carries zero information about it: full workspace build, then --re-measure re-ran tsc per ledger entry — none above its recorded number, the package's recorded 63 unchanged, so the new test file contributes 0 errors. No ceiling raised.


Verified at b1de760d6 (post-merge head; the union above was run at this sha).

Generated by Claude Code


Generated by Claude Code

…taDiagnostics (#8855)
The per-type read in `getMetaDiagnostics` was wrapped in an untyped `catch`
that `continue`d. The comment named a benign reason ("type not listable in
this kernel scope") and that reason is real — but the catch took everything
else with it, including the 503 `getMetaItems` raises for a `sys_metadata`
read that failed for any reason other than "not provisioned yet" (#5532).
Swallowing it put the emptiness back one layer up and published it as a
number: `stats[t]` was never written (the type ABSENT from the response, not
zero — the Studio directory tile simply loses it) and `total`, which counts
entries that failed validation, read 0 because an unreadable store contributes
none. A diagnostics endpoint reported "0 problems" precisely when it could
read nothing.
The catch is narrowed, not deleted: a 503 is rethrown unchanged, every other
failure still skips that one type. No response field is added. The envelope is
propagated rather than re-classified, so the driver error stays on `cause`.
ADR-0110 D3 — a miss and an outage are different facts with opposite
dispositions, and a consumer must never read one as the other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XeQRiAa7vYRVX5Fog7Zby8
@vercel

vercelBot commented Aug 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 15, 2026 5:06pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-protocol.

3 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/metadata-lifecycle.mdx(via @objectstack/metadata-protocol)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/metadata-protocol)
  • content/docs/protocol/kernel/http-protocol.mdx(via @objectstack/metadata-protocol)

1 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/v9.mdx(via @objectstack/metadata-protocol)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

2 participants

@hotlong@claude