Uh oh!
There was an error while loading. Please reload this page.
fix(spec): regenerate drifted skills output and gate it in CI - #3143
Closed
os-zhuang wants to merge 1 commit into
Closed
fix(spec): regenerate drifted skills output and gate it in CI#3143os-zhuang wants to merge 1 commit into
os-zhuang wants to merge 1 commit into
Conversation
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 102 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
`skills/` is generated from the spec by `gen:skill-refs` and `gen:react-blocks` and committed, but neither had a `check:` sibling and nothing in CI regenerated them. Both existing gates miss this surface: `check:docs` (#3134) covers only content/docs/references/, and `check:skill-docs` only SKILL.md frontmatter → README + skills-reference.mdx. So it drifted — 8 committed files were stale on main. This matters more than the content/docs case: `skills/` is published to third parties via `npx skills add objectstack-ai/framework --all`, so stale output ships to users. Fix the drift's root cause, not the output ------------------------------------------ Two published skills pointed agents at `data/dataset.zod.ts`, which #1620 renamed to `data/seed.zod.ts`. The generator warned and skipped, staying green, so objectstack-data and objectstack-platform silently lost their seed schema pointer and told agents to Read a path that no longer exists. SKILL_MAP now names seed.zod.ts, and a missing mapped file is a hard error instead of a warning — that half-rename is exactly what warn-and-skip is good at hiding. Add --check to both generators ------------------------------ Following the convention #3134 established in build-docs.ts: every write goes through emit(), every regenerated folder through manageDir(), and nothing touches the tree until flush() — so check and write share byte-for-byte identical generation logic and differ only in final disposition. Not `git diff --exit-code`, which sees only tracked files and would miss a new or leftover page. The sink is extracted to scripts/lib/generated-output.ts since both generators need it; manageDir() takes a predicate because a skill's references/ folder is a selective wipe, not a wholesale one — hand-written siblings (data-hooks.md, plugin-hooks.md) and the other generator's react-blocks.md live there and must survive. Wire into "TypeScript Type Check", not ESLint --------------------------------------------- main requires exactly 4 status checks (TypeScript Type Check, Build Core, Test Core, Dogfood Regression Gate). ESLint is not one of them, so a gate placed there runs but cannot block a merge. Same job as check:docs, same reason: no paths filter, required, and both read src/ via tsx with no build (~2s). Proven able to go red before shipping: tampered content, deleted file, leftover file, leftover in a sub-folder, stale SKILL_MAP entry, and both vacuous cases (empty SKILL_MAP / empty REACT_BLOCKS) each exit 1; a hand-written .md dropped in references/ stays green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
xuyushun441-sysforce-pushed
the
claude/inspiring-dubinsky-f1d663
branch
from
July 17, 2026 13:22
01ed191 to
68a6a59Compareos-zhuang
commented
Jul 17, 2026
ContributorAuthor
Heads-up: we collided. #3138 was landed on The generated output is fully redundant now. I diffed all 8 of this PR's Byte-identical — including the same We also independently reached the same three conclusions, which I'd take as corroboration of the design:
What's on If anything in your implementation is better than what landed, it's worth salvaging as a follow-up on top of |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
skills/is generated from the spec bygen:skill-refsandgen:react-blocksand committed, but neither generator had acheck:sibling and nothing in CI regenerated them. Both existing gates miss this surface:check:docs(fix(spec): regenerate drifted reference docs and gate them in CI #3134) covers onlycontent/docs/references/check:skill-docscovers only SKILL.md frontmatter → README +skills-reference.mdxSo it drifted: 8 committed files were stale on main (verified on
c22e2c3fc).This bites harder than the content/docs case —
skills/is published to third parties vianpx skills add objectstack-ai/framework --all, so stale output ships to users.1. A real bug, not just drift
Two published skills pointed agents at
data/dataset.zod.ts— renamed todata/seed.zod.tsby #1620 ("complete Dataset->Seed rename"). The generator warned and skipped, staying green, soobjectstack-dataandobjectstack-platformsilently lost their seed schema pointer and told agents toReada path that does not exist.Fixed at the source (
SKILL_MAP), not in the output. A missing mapped file is now a hard error rather than a warning — warn-and-skip is exactly what let this half-rename hide.The rest of the diff is legitimate spec drift: a
drawerWidthdeprecation, a newrelationshipValueFieldprop,columnsbecoming optional.2.
--checkon both generatorsFollows the convention #3134 established in
build-docs.ts: every write goes throughemit(), every regenerated folder throughmanageDir(), and nothing touches the tree untilflush()— so check and write run byte-for-byte identical generation logic and differ only in final disposition. Notgit diff --exit-code, which only sees tracked files and would miss a new or leftover page.The sink is extracted to
scripts/lib/generated-output.tssince both generators need it.manageDir()takes a predicate because a skill'sreferences/folder is a selective wipe, not a wholesale one — hand-written siblings (data-hooks.md,plugin-hooks.md) and the other generator'sreact-blocks.mdlive there and must survive.build-docs.tskeeps its own inline copy for now; it predates the extraction and is under active change (#3126), so migrating it belongs in its own PR.3. Wired into "TypeScript Type Check", not ESLint
mainrequires exactly 4 status checks (TypeScript Type Check, Build Core, Test Core, Dogfood Regression Gate). ESLint is not one of them, so a gate placed there runs but cannot block a merge. (Re-verified: the one ruleset isenforcement: disabledand adds no checks.) Same job ascheck:docs, same reason: no paths filter, required, and both readsrc/via tsx with no build — they fail in ~2s.Why not the existing "Check Generated Artifacts" job?
That job in
ci.ymllooks like the natural home — it already runscheck:skill-docs. But it is the wrong place twice over, and its own filter comments document this class of bug biting twice before:generatedfilter watchesskills/**andbuild-skill-docs.ts, but notpackages/spec/src/**. Every file in this PR's drift traces to apackages/spec/src/**change (theseed.zod.tsrename,view.zod.ts'sdrawerWidthdeprecation, the newrelationshipValueField). A spec-only PR changes the generated output without touchingskills/, so the filter would not match and the job would skip — which is exactly how this drift shipped.check:docslanded in the typecheck job for precisely this reason (#3134), and its comment says so. These gates follow it.Proving the gate can go red
A gate that cannot fail is theatre, so each failure mode was exercised — all 14 as expected:
references/SKILL_MAP→ missing skill dirSKILL_MAP→ renamed/missing zod fileSKILL_MAPREACT_BLOCKSreact-blocks.mdcontracts/.mdinreferences/The vacuous guards matter because a run that finds no input emits nothing, and "nothing differs" would read as success — the gate would pass while checking nothing at all. Both now fail loudly instead of greenly. The negative control proves the selective-wipe predicate is not over-broad.
Also verified: the original repro (
gen:skill-refs && gen:react-blocks && git status skills/) now produces zero drift;check:docs,check:role-word(bidirectional ratchet, scopesskills/),check:doc-authoring, andcheck:nul-bytesall stay green.🤖 Generated with Claude Code