Skip to content

chore(spec): spec liveness gate — classify-or-fail for authorable properties - #1919

Merged
os-zhuang merged 1 commit into
mainfrom
chore/spec-liveness-gate
Jun 15, 2026
Merged

chore(spec): spec liveness gate — classify-or-fail for authorable properties#1919
os-zhuang merged 1 commit into
mainfrom
chore/spec-liveness-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

First step of the long-term "close the spec↔runtime gap" direction. For a metadata-driven platform the spec is the product surface — a parsed-but-unenforced property is a silent no-op, and for a security property a silent no-op is false compliance (e.g. forceMfa: true accepted and ignored). The metadata-liveness audits found large DEAD swaths; this makes the classification explicit and regression-proof.

What it does

Every authorable property in a governed category must declare a runtime-liveness status with evidence in packages/spec/liveness/<category>.json, or CI fails — the ratchet: no new undeclared surface.

StatusMeaning
livehas a runtime consumer (cite file:line)
experimental / planneddeclared, intentionally not enforced (also read from [EXPERIMENTAL — not enforced] spec markers)
deadparsed, no consumer → enforce-or-remove worklist
internalruntime DTO/result, not authorable (exempt)

Resolution: ledger entry → spec .describe() marker → UNCLASSIFIED.

Seeded: security (the P0 category)

93 authorable properties — 66 dead, 26 live, 1 experimental. ~71% of the authorable security surface is parsed-but-unenforced. Seeded from docs/audits/2026-06-security-identity-property-liveness.md (file:line evidence) + greps for what the audit didn't cover. The dead set is the concrete worklist for the security enforce-or-remove ADR — most urgently the ungated destructive ObjectPermission.allow{Transfer,Restore,Purge} and the entirely-dead Policy tree (password/session/forceMfa/network/audit).

Pieces

  • packages/spec/scripts/liveness/check-liveness.mjs — the gate (--dump, --json).
  • packages/spec/liveness/security.json — the seeded ledger.
  • .github/workflows/spec-liveness-check.yml — runs on PRs touching packages/spec/**.
  • pnpm --filter @objectstack/spec check:liveness + packages/spec/liveness/README.md.

Verified

  • Gate green on security (0 unclassified, exit 0).
  • Ratchet fires: injecting a new ObjectPermission flag → unclassified → exit 1.
  • json-schema/ is generated (gitignored); CI regenerates via gen:schema before checking.

Rollout

Governed today: security only. Other categories (data, automation, ui, …) are added one at a time, highest-risk-first, each seeded from its existing audit — see the README. Same "drift → CI gate" pattern as the docs-accuracy system (#1906).

🤖 Generated with Claude Code

…perties
For a metadata-driven platform the spec IS the product surface; a parsed-but-
unenforced property is a silent no-op, and for security props a silent no-op is
false compliance (e.g. forceMfa accepted and ignored). The metadata-liveness
audits found large dead swaths. This makes the classification explicit and
regression-proof.
- packages/spec/scripts/liveness/check-liveness.mjs — reads the generated
json-schema/<category>/*.json, resolves each authorable property's liveness
(ledger entry > spec .describe() marker > UNCLASSIFIED), and exits non-zero on
any unclassified property in a GOVERNED category (the ratchet: no new
undeclared surface). --dump inventories a category; --json for machines.
- packages/spec/liveness/security.json — security ledger seeded from
docs/audits/2026-06-security-identity-property-liveness.md (file:line evidence)
plus greps for schemas the audit didn't cover. 93 props: 66 dead, 26 live,
1 experimental. The dead set (Policy tree, allow{Transfer,Restore,Purge},
isProfile, contextVariables, SharingRule, Territory, RLSConfig) is the
enforce-or-remove worklist.
- .github/workflows/spec-liveness-check.yml — runs the gate on PRs touching
packages/spec/** (gen:schema then check).
- check:liveness npm script + packages/spec/liveness/README.md (how to roll out
the next category, highest-risk-first).
Governed today: security only. Other categories are added one at a time as their
ledgers are seeded from the existing audits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented Jun 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
specReadyReadyPreview, CommentJun 15, 2026 3:18pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

89 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/architecture.mdx(via @objectstack/spec)
  • content/docs/concepts/cloud-artifact-api.mdx(via packages/spec)
  • content/docs/concepts/cluster-semantics.mdx(via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx(via packages/spec)
  • content/docs/concepts/implementation-status.mdx(via @objectstack/spec)
  • content/docs/concepts/index.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx(via packages/spec)
  • content/docs/concepts/north-star.mdx(via packages/spec)
  • content/docs/concepts/packages.mdx(via @objectstack/spec)
  • content/docs/concepts/setup-app.mdx(via @objectstack/spec)
  • content/docs/concepts/skills.mdx(via @objectstack/spec)
  • content/docs/concepts/webhook-delivery.mdx(via @objectstack/spec)
  • content/docs/getting-started/architecture.mdx(via @objectstack/spec)
  • content/docs/getting-started/cli.mdx(via @objectstack/spec)
  • content/docs/getting-started/core-concepts.mdx(via @objectstack/spec)
  • content/docs/getting-started/examples.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx(via @objectstack/spec)
  • content/docs/guides/adding-a-metadata-type.mdx(via @objectstack/spec)
  • content/docs/guides/ai-capabilities.mdx(via @objectstack/spec)
  • content/docs/guides/airtable-dashboard-analysis.mdx(via @objectstack/spec)
  • content/docs/guides/analytics-datasets.mdx(via @objectstack/spec)
  • content/docs/guides/api-reference.mdx(via @objectstack/spec)
  • content/docs/guides/business-logic.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/backward-compatibility.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/error-catalog.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-type-gallery.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-validation-rules.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/permissions-matrix.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/protocol-diagram.mdx(via packages/spec)
  • content/docs/guides/cheatsheets/query-cheat-sheet.mdx(via @objectstack/spec)
  • content/docs/guides/cheatsheets/quick-reference.mdx(via @objectstack/spec)
  • content/docs/guides/client-sdk.mdx(via @objectstack/spec)
  • content/docs/guides/common-patterns.mdx(via @objectstack/spec)
  • content/docs/guides/contracts/auth-service.mdx(via packages/spec)
  • content/docs/guides/contracts/cache-service.mdx(via packages/spec)
  • content/docs/guides/contracts/data-engine.mdx(via @objectstack/spec)
  • content/docs/guides/contracts/index.mdx(via @objectstack/spec)
  • content/docs/guides/contracts/metadata-service.mdx(via packages/spec)
  • content/docs/guides/contracts/storage-service.mdx(via packages/spec)
  • content/docs/guides/data-modeling.mdx(via @objectstack/spec)
  • content/docs/guides/deployment-vercel.mdx(via @objectstack/spec)
  • content/docs/guides/driver-configuration.mdx(via @objectstack/spec)
  • content/docs/guides/error-handling-client.mdx(via @objectstack/spec)
  • content/docs/guides/error-handling-server.mdx(via @objectstack/spec)
  • content/docs/guides/formula.mdx(via @objectstack/spec)
  • content/docs/guides/hook-bodies.mdx(via packages/spec)
  • content/docs/guides/kernel-services.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/dashboard.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/field.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/flow.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/index.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/object.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/validation.mdx(via @objectstack/spec)
  • content/docs/guides/metadata/workflow.mdx(via @objectstack/spec)
  • content/docs/guides/packages.mdx(via @objectstack/spec)
  • content/docs/guides/plugin-development.mdx(via @objectstack/spec)
  • content/docs/guides/plugins.mdx(via @objectstack/spec)
  • content/docs/guides/project-scoping.mdx(via @objectstack/spec)
  • content/docs/guides/public-forms.mdx(via @objectstack/spec)
  • content/docs/guides/runtime-services/email-service.mdx(via packages/spec)
  • content/docs/guides/runtime-services/index.mdx(via packages/spec)
  • content/docs/guides/runtime-services/queue-service.mdx(via packages/spec)
  • content/docs/guides/runtime-services/sharing-service.mdx(via packages/spec)
  • content/docs/guides/runtime-services/storage-service.mdx(via packages/spec)
  • content/docs/guides/security.mdx(via @objectstack/spec)
  • content/docs/guides/seed-data.mdx(via @objectstack/spec)
  • content/docs/guides/skills.mdx(via @objectstack/spec)
  • content/docs/guides/standards.mdx(via @objectstack/spec)
  • content/docs/guides/troubleshooting.mdx(via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx(via @objectstack/spec)
  • content/docs/protocol/objectos/config-resolution.mdx(via @objectstack/spec)
  • content/docs/protocol/objectos/i18n-standard.mdx(via @objectstack/spec)
  • content/docs/protocol/objectos/lifecycle.mdx(via @objectstack/spec)
  • content/docs/protocol/objectos/plugin-spec.mdx(via @objectstack/spec)
  • content/docs/protocol/objectos/runtime-capabilities.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx(via packages/spec)
  • content/docs/protocol/objectql/query-syntax.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx(via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx(via packages/spec)
  • content/docs/protocol/objectui/record-alert.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx(via @objectstack/spec)
  • content/docs/releases/index.mdx(via @objectstack/spec)
  • content/docs/releases/v9.mdx(via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tooling labels Jun 15, 2026
@os-zhuang
os-zhuang merged commit 43ecc08 into mainJun 15, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the chore/spec-liveness-gate branch June 15, 2026 15:39
os-zhuang added a commit that referenced this pull request Jun 18, 2026
…2024)
* docs(adr): ADR-0054 prove-it-runs gate for the authorable surface
Extends ADR-0049 (enforce-or-remove) with a third leg. The liveness ledger
(#1919) classifies every authorable property live/experimental/dead, but "live"
means only a static file:line consumer pointer — proof that something reads the
property, not that authoring it produces correct runtime behavior. #2018 (tz
bucketing: live at every layer, broken in integration) and the field-type
fidelity gaps (#2022: rating/slider/toggle read back wrong-typed) fell through
that gap — call it "unproven liveness".
For a platform whose authors are AI emitting metadata across a combinatorial
space the examples never cover, unproven liveness ships silently into
third-party apps. ADR-0054 upgrades a `live` classification to optionally carry a
`proof` — a @objectstack/dogfood test that authors the property against the real
in-process stack and asserts the runtime outcome. Required as a ratchet (not a
retrofit) for a high-risk authorable class on change, and for any property
implicated in a shipped regression (the fix carries its proof). Generative
testing is explicitly deferred (Phase 3, evidence-gated).
Proposed — for architect review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* docs(adr): accept ADR-0054 (prove-it-runs gate)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
os-zhuang added a commit that referenced this pull request Jun 23, 2026
fix(grid): rows-per-page selector honors pagination.pageSizeOptions; drop duplicate ListView <select> (#1919)
objectui@92c32428ff6ff1e488e0c233777db654233afede
@github-actionsgithub-actionsBot mentioned this pull request Jun 24, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filedocumentationImprovements or additions to documentationsize/mtooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@os-zhuang