Skip to content

feat(spec): compile-check skills/ TypeScript examples (anti-drift, #3094) - #3224

Merged
os-zhuang merged 1 commit into
mainfrom
ci/skill-example-drift-gate
Jul 18, 2026
Merged

feat(spec): compile-check skills/ TypeScript examples (anti-drift, #3094)#3224
os-zhuang merged 1 commit into
mainfrom
ci/skill-example-drift-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes#3094.

What

Adds check:skill-examples — a CI gate that type-checks the TypeScript examples inside skills/ against the built @objectstack/spec, so a renamed export or tightened union fails here instead of in a third party's editor. This is the "示例不漂移" layer; the existing check:skill-refs / check:skill-docs only cover the "目录不漂移" (generated reference indexes).

Design

  • Opt-in by marker, not fence-meta. A self-contained, should-compile block is tagged with an inert <!-- os:check --> HTML comment on the line directly above its fence. Full extraction of every ```ts block is infeasible — most are fragments (a columns: [...] subtree) that need a hand-authored wrapper and produce false-positive noise.
    • Deliberately not a ```ts check fence-meta: that would leave the fence info-string non-standard and punch a hole in the existing check:doc-authoring scanner (which keys on ^\``(ts|typescript|tsx)$). The comment keeps the fence a bare ```ts `, so every existing scanner still sees the block.
  • Consumer-faithful resolution. Extracted blocks compile against the built dist/*.d.ts via a tsconfig paths map derived from the spec's own exports field — the exact surface import { … } from '@objectstack/spec' resolves to for a consumer, and it self-updates as subpath exports change.
  • Placement. Reads the built dist, so it runs in the required TypeScript Type Check job after the build step, next to its fellow "real consumer" gates (check:api-surface, downstream-contract, example-app typecheck) — not before the build like check:skill-refs.
  • Error mapping. tsc diagnostics are remapped from the throwaway build file back to skills/**/SKILL.md:<real line>.

Proven-red (門必先证明能红)

Failure modeResult
Type error in a tagged block✗ (caught the 8 real drifts below)
Spec not built (no dist/*.d.ts)✗ loud guard, never vacuous
Zero marked blocks (marker stripped/renamed)✗ vacuous-green guard
Orphan marker (not directly above a ts fence)✗ misplaced-marker guard

Drift the gate immediately surfaced (fixed here)

Tagging 19 self-contained examples turned up real, shipping drift:

  • objectstack-dataObjectSchema imported from the package root; it's only exported from /data (×2).
  • objectstack-platform — the removed Data namespace (import { Data } … ; const { Field } = Data) → import { Field } from '@objectstack/spec/data' (×2).
  • objectstack-ui — a defineAction example using P`…` without import { P }.

Non-self-contained fragments (local-module imports like ./apps/crm/objectstack.config, external object refs like Opportunity) are left untagged by design — they can't compile standalone and aren't meant to.

Deferred (documented, not silently dropped)

The platform feature-flags example uses a top-level featureFlags key that isn't in ObjectStackDefinitionInput — feature flags live in the kernel capability config (features: FeatureFlagSchema[], nested), and environment is a single enum (dev|staging|prod|all), not an array. Correcting it is a non-trivial rewrite of that section, so the block is left exactly as authored and untagged, to be fixed in a follow-up rather than shipping a guessed correction.

Growing coverage

New self-contained examples opt in by adding <!-- os:check --> above their fence. A misplaced marker errors (orphan guard), so under-coverage can't hide.

…stack/spec (#3094)
The TypeScript in skills/ is the first thing an AI copies when authoring
metadata, yet nothing type-checked it, so it rotted silently. A new
`check:skill-examples` gate extracts every code block tagged with an inert
`<!-- os:check -->` comment and runs it through `tsc --noEmit` against the
built spec declarations — the exact surface a consumer's import resolves to.
Opt-in by marker (not a `` ```ts check `` fence-meta) so the fence info-string
stays a bare ``` ```ts ``` and the existing `check:doc-authoring` scanner keeps
seeing the block. Module resolution is a `paths` map derived from the spec's own
`exports`, so it self-updates as subpath exports change. Runs after the build
step in the required `TypeScript Type Check` job, alongside the other
"real consumer" gates (api-surface, downstream-contract, example-app typecheck).
Four provable red modes: a type error in a tagged block, spec not built,
zero marked blocks (vacuous-green guard), a misplaced/orphan marker.
Tagging 19 self-contained examples surfaced real drift, now fixed:
- objectstack-data: `ObjectSchema` imported from the root instead of `/data` (×2)
- objectstack-platform: the removed `Data` namespace (`const { Field } = Data`)
→ `import { Field } from '@objectstack/spec/data'` (×2)
- objectstack-ui: a `defineAction` example using `P` without importing it
Non-self-contained fragments (local-module imports, external object refs) are
left untagged by design. One deeper drift is deferred: the platform feature-flags
example uses a top-level `featureFlags` key absent from `ObjectStackDefinitionInput`
(flags live in the kernel capability config) — left as-authored, untagged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercelBot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
specReadyReadyPreview, CommentJul 18, 2026 3:08pm

Request Review

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tooling labels Jul 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

102 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx(via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx(via @objectstack/spec)
  • content/docs/ai/skills.mdx(via @objectstack/spec)
  • content/docs/api/client-sdk.mdx(via @objectstack/spec)
  • content/docs/api/environment-routing.mdx(via @objectstack/spec)
  • content/docs/api/error-catalog.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx(via @objectstack/spec)
  • content/docs/api/index.mdx(via @objectstack/spec)
  • content/docs/automation/approvals.mdx(via packages/spec)
  • content/docs/automation/flows.mdx(via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx(via packages/spec)
  • content/docs/automation/hooks.mdx(via @objectstack/spec)
  • content/docs/automation/index.mdx(via @objectstack/spec)
  • content/docs/automation/webhooks.mdx(via @objectstack/spec)
  • content/docs/automation/workflows.mdx(via @objectstack/spec)
  • content/docs/concepts/architecture.mdx(via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx(via packages/spec)
  • content/docs/concepts/index.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx(via packages/spec)
  • content/docs/concepts/north-star.mdx(via packages/spec)
  • content/docs/data-modeling/analytics.mdx(via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx(via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx(via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx(via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx(via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx(via @objectstack/spec)
  • content/docs/data-modeling/index.mdx(via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx(via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx(via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx(via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx(via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx(via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx(via @objectstack/spec)
  • content/docs/getting-started/cli.mdx(via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx(via @objectstack/spec)
  • content/docs/getting-started/examples.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx(via @objectstack/spec)
  • content/docs/getting-started/validating-metadata.mdx(via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx(via @objectstack/spec)
  • content/docs/kernel/cluster.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx(via packages/spec)
  • content/docs/kernel/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx(via packages/spec)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/spec)
  • content/docs/permissions/authorization.mdx(via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx(via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx(via @objectstack/spec)
  • content/docs/permissions/positions.mdx(via @objectstack/spec)
  • content/docs/permissions/rls.mdx(via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx(via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx(via @objectstack/spec)
  • content/docs/plugins/development.mdx(via @objectstack/spec)
  • content/docs/plugins/index.mdx(via @objectstack/spec)
  • content/docs/plugins/packages.mdx(via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx(via @objectstack/spec)
  • content/docs/protocol/diagram.mdx(via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx(via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx(via packages/spec)
  • content/docs/protocol/objectql/query-syntax.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx(via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx(via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx(via @objectstack/spec)
  • content/docs/releases/index.mdx(via @objectstack/spec)
  • content/docs/releases/v12.mdx(via @objectstack/spec)
  • content/docs/releases/v13.mdx(via @objectstack/spec)
  • content/docs/releases/v9.mdx(via @objectstack/spec)
  • content/docs/ui/actions.mdx(via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx(via @objectstack/spec)
  • content/docs/ui/dashboards.mdx(via @objectstack/spec)
  • content/docs/ui/forms.mdx(via @objectstack/spec)
  • content/docs/ui/index.mdx(via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx(via @objectstack/spec)
  • content/docs/ui/setup-app.mdx(via @objectstack/spec)
  • content/docs/ui/translations.mdx(via @objectstack/spec)
  • content/docs/ui/views.mdx(via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuangos-zhuang added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Jul 18, 2026
@os-zhuang
os-zhuang merged commit dccffdb into mainJul 18, 2026
21 checks passed
@os-zhuang
os-zhuang deleted the ci/skill-example-drift-gate branch July 18, 2026 16:14
@os-zhuang

Copy link
Copy Markdown
ContributorAuthor

Follow-up for the deferred feature-flags drift (noted in the PR description): #3248 — the platform featureFlags example teaches a top-level key that isn't in ObjectStackDefinitionInput; needs a product/spec decision on the real authoring surface before it can be rewritten and re-tagged with <!-- os:check -->.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filedocumentationImprovements or additions to documentationsize/mskip-changesetPR has no user-facing published change; bypasses the changeset gatetooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI gate: compile-check the TypeScript examples inside skills/ (anti-drift)

1 participant

@os-zhuang