Skip to content

[finding] YAML doc fences are validated by nothing — check:skill-examples reads only ts/tsx fences, and only opt-in, so every metadata example in content/docs/** is unverifiable by construction #13086

Description

@os-elon

Found while fixing #12935 (the field-level visible key taught in three shapes that
no schema declares). Filed unassigned, recording only. This is the mechanism half of
that card: not another phantom key, but the reason phantom keys keep surviving sweeps.

Measured on origin/main @ db39dfc1c9b09681474475cea6e77537ab2c89ab.

What was measured

packages/spec/scripts/check-skill-examples.ts is the one gate that compiles doc prose
against the live spec. Two properties, both deliberate and both documented in its own
header, together mean it can never see the defect class #12935 belongs to:

  1. It reads only TypeScript fences. Its FENCE_OPEN_RE recognises ts, tsx and
    typescript and nothing else. A yaml fence is not in the population at all — not
    as a skipped block, not as an unchecked one. It does not exist to the gate.
  2. It is opt-in. Even inside the TS population, only blocks carrying an explicit
    os:check marker are compiled. Its last run over this tree reports
    260 marked example(s) across 101 file(s), 3 surface(s).

The consequence is structural rather than incidental: metadata authoring examples are
written in YAML.
That is the whole authoring surface this documentation teaches — object
definitions, view layouts, sections, fields, actions. So the format that carries almost
every example a metadata author copies is precisely the format no gate reads.

Cross-check on today's tree: content/docs/protocol/objectui/layout-dsl.mdx carries an
os:check marker at its "Visibility Rule" section over a typescript fence, which is
type-checked — while the yaml fences on the same page taught a field-level visible
breakpoint map and a section-level columns orientation map, neither of which exists on
any schema, for as long as the page has existed.

Why it is worth recording

Prior art, deliberately not duplicated

#11407 recorded a different limit of the same gate — that an os:check fence
type-checks a visibleWhen CEL string as string, so any predicate text passes. That is
about the depth of checking inside a block the gate already reads. This is about the
population the gate reads at all.

Not a decision

Recording only, and the shape of any fix is genuinely open — a YAML-fence checker keyed
to the metadata type an example declares is one option; an opt-in marker over yaml
fences mirroring os:check is another and much cheaper; declaring the whole class
unverifiable and moving authoring examples to ts fences is a third. Each has a real
cost and none of them is obviously right, so this is left for triage rather than
pre-judged.

Re-check commands:

  • git grep -n "FENCE_OPEN_RE" -- packages/spec/scripts/check-skill-examples.ts
  • pnpm --filter @objectstack/spec run check:skill-examples (reports its marked-block census)
  • git grep -c "os:check" -- content/docs versus the count of ```yaml fences on the same pages

Generated by Claude Code

Metadata

Metadata

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions