Uh oh!
There was an error while loading. Please reload this page.
skills(platform): delete the placeholder eval README and drop its ceiling row (#14296 item 2) - #14681
Merged
Merged
Conversation
…ling row The file was a stub for evals that were never written: it says the directory is "reserved for future skill evaluation tests", that "When implemented, evals will follow this structure", and that it is "Not yet implemented — This is a placeholder for future development". The tree it prints names ten eval files, none of which exists anywhere in the repository. Nothing read the path. Repo-wide, the single reference to skills/objectstack-platform/evals was its own ceiling row in the token ratchet, so the row goes with the file and the package ceiling steps down by 514 tokens. No other row is touched, no ceiling is raised, and no prose changes. Verified before the deletion (the ruling keeps real fixtures, so the directory had to be proven fixture-free): the evals directory held README.md and nothing else, with zero subdirectories. Reverse verification: with the file deleted and the row still present, check-skills-token-ratchet exits 1 with "cannot read skills/objectstack-platform/evals/README.md — red, not a skip (#4690)"; with the row dropped it exits 0 over 30 authored bundle files; --self-test passes 64 cases. Ruled A for item 2 of the eval-stub decision card (#14296): planned-eval stubs are deleted, their ceilings step down, real fixtures stay, no new eval packages. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
…atform-eval-stub-removal
This was referenced Sep 2, 2026
os-zhuang
approved these changes
Sep 2, 2026
os-zhuang
marked this pull request as ready for review
September 2, 2026 22:52
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#14299
Ruling (not re-adjudicated here). Item 2 of the eval-stub decision card #14296 was ruled A: planned-eval stubs are deleted, their ceilings step down, real fixtures stay, and no new eval packages are written. This is round 2 of #14299 — finding PLAT-H-01, the item round 1 (PR #14351, landed as
Part of #14299) deferred. It is this card's last in-scope item: the other deferred finding, PLAT-G-02 (thereferences/placement question), is carried by the catalog-wide card #14655, which is why the first line here is a closing keyword and round 1's was not.Same two-file shape as the merged precedents #14585 (query) and #14578 (automation), and as #14652 (ai).
Pre-deletion verification
The ruling keeps real fixtures, so the directory had to be proven fixture-free before anything was touched. Three readings, all on the branch base
20b88391:find skills/objectstack-platform/evalsreturned the directory andREADME.md— nothing else, zero subdirectories.:3"This directory is reserved for future skill evaluation tests";:11"When implemented, evals will follow this structure";:41":13-29names ten eval files; each was searched for by name across the whole tree and all ten return 0 matches —test-definestack-keys,test-manifest-required-fields,test-template-selection,test-driver-selection,test-hono-integration,test-lifecycle-phases,test-service-registry,test-kernel-events,test-cli-commands,test-litekernel-testing.git grep -n "objectstack-platform/evals"returns exactly one hit repo-wide —scripts/check-skills-token-ratchet.mjs:420, the ceiling row itself. The package linksevalsfrom nowhere:git grep -n evals -- skills/objectstack-platform/returns only lines inside the deleted file.No real fixture was found, so the ruling's precondition holds and the stub is deletable.
The diff — two files, 50 deletions, nothing else
skills/objectstack-platform/evals/README.mdscripts/check-skills-token-ratchet.mjs['skills/objectstack-platform/evals/README.md', 514],)Proved mechanically rather than asserted: the
CEILINGSmap parsed out of the base blob and out of the head file gives 31 rows / 165,532 ceiling tokens before → 30 rows / 165,018 after; removed keys = exactlyskills/objectstack-platform/evals/README.md (514); added keys = none; rows whose value changed = none. No other row moved, no ceiling was raised, no row was added, and no prose was edited anywhere.The
// objectstack-platformsection comment above the row is left standing and untouched, per the flight constraint. It is not orphaned by this change: three rows remain under it (references/plugin-hooks.md,rules/plugin-lifecycle.md,rules/service-registry.md).Token readings for the governed
skills/**surfaceevals/README.md)skills/objectstack-platformPure deletion — nothing was added, so there is no expansion to budget against.
514 = ceil(2055 utf8 bytes / 4), the ratchet's own price convention, and the file sat at its ceiling with headroom 0.Reverse verification — red, then green
Both readings taken with the exit code captured by redirect-then-capture, never after a pipe.
Leg 1 — file deleted, ceiling row still present ⇒ the gate must go RED, not skip.
node scripts/check-skills-token-ratchet.mjs,EXIT=1, and it printed its documented missing-input line verbatim:Leg 2 — row dropped ⇒ green. Same command,
EXIT=0:Leg 3 — the script's own test suite.
node scripts/check-skills-token-ratchet.mjs --self-test,EXIT=0:The gate script is edited by this diff, so its own suite is owed on top of the derived family.
--self-testis that suite: no vitest file names this script (git grep -l check-skills-token-ratchetover test globs returns nothing), and the three other scripts that do read it —check:ratchet-remedy-authority,check:doc-authoring,scripts/pm/bare-root-worklist.mjs --self-test— are all in the derived family below and all exit 0.Gates
Head sha
a8ceb57b— the merge commit, and the head every reading below was taken on.origin/mainmoved from the branch base20b88391to2aa8456cwhile this was in flight, so it was merged in, not rebased; the merge was conflict-free, and neither the hot file nor any file of this package had moved on main. The PR's own delta is unchanged by the merge: 2 files, 50 deletions.Family derived from the real change set at the merge head, not hand-written:
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands⇒ 24 families (22 by path + 2 by change kind — this diff edits a gate script). Byte-identical to the list derived before the merge. All 24 run throughscripts/pm/os-verify-lock.sh, slotissue-14299-r2; every exit code captured before any pipe.22 of 24 exit 0, plus
check:skill-docsandcheck:nul-bytesrun beyond the family. 2 of 24 are NOT MEASURED (exit 3), both by their own prerequisite text.node scripts/check-skills-token-ratchet.mjs✓ 30 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted.pnpm check:ratchet-remedy-authorityOK 184 scripts swept; 12 mark the expanding remedy ⛔ MAINTAINER-ONLY, 6 turn it down outright, 166 hand out no ratchet-expanding remedy.node scripts/pm/bare-root-worklist.mjs --self-testOK 57 live row(s), 49 unreachable as spelled, 49 recorded verdict(s) — none stale, none missing, none contradicted.pnpm check:pm-dispatch-gates✓ dispatch-gates self-test: 1240 cases pass.pnpm check:skill-identifier-livenessOK — Leg 1: 465 citation(s) over 40 published file(s) ... Leg 2: 8 registered exhaustive section(s), 0 ledgered gap(s).pnpm check:skill-compatibility✓ 11 SKILL.md file(s) reconciled against 79 workspace packagespnpm --filter @objectstack/spec check:skill-docs✅ Skill docs in sync— no regeneration demanded; this diff touches no frontmatterpnpm check:skill-frame-syncbinding sentence present in all 4; 4 count mention(s) agree; 34 markdown files scannedpnpm check:corpus-claim-driftpnpm check:doc-authoring✓ 14500 customer-facing string(s) across 710 spec sources cleanpnpm check:pm-governed-merges✓ --self-test: 243 assertions+live: the real generator declared 9 output(s) and certified this treepnpm check:watch-hint-literal✓ 45 declaration(s) across 4 rostered name(s) ... no unrostered spelling of the idiom in the tree.pnpm check:role-wordpnpm check:agent-test-spellingpnpm check:bash32-floorpnpm check:cli-command-idspnpm check:cross-package-test-inputsnode scripts/check-cross-package-test-inputs.mjsnode scripts/check-ci-filter-parity.mjsnode scripts/check-shard-attestation.mjspnpm check:entry-guardpnpm check:parse-guardpnpm check:pnpm-filter-targetspnpm check:nul-bytes(beyond the family)OK (scanned 7994 text file(s) ... no raw ASCII control bytes), plus a direct control-byte scan of the edited script: no hitsNOT MEASURED (exit 3 — a prerequisite this flight does not supply, never a red)
node scripts/check-test-completeness.mjsPREREQUISITE NOT MET — this gate grades a saved 'turbo run test' log, and no log was named.pnpm --filter @objectstack/lint run check:doc-formula-expressionsPREREQUISITE NOT MET — the workspace package @objectstack/formula is not builtNeither is touched by this diff — it adds no test and no formula expression — and both print, in their own words, that the local result "says NOTHING about what it gates ... It is NOT a finding". Recorded as NOT MEASURED, never counted as green and never called a flake.
Housekeeping
skip-changesetis applied — checked againstscripts/check-empty-changeset.mjs's own enumeration rather than by habit. That enumeration reads "It releases nothing (.github/, .claude/, skills/, docs/, content/, examples/, tests-only, and the like) -> delete the changeset and apply the 'skip-changeset' label (route 2)":skills/is named verbatim, andscripts/check-skills-token-ratchet.mjsis a CI-internal gate script that publishes from no package, which is the "and the like" case. The same file pins askills/**-only PR carrying no changeset at all as a green fixture (its GREEN 3 self-test case).needs:contract-review: a deletion of a placeholder that documents nothing about runtime behaviour makes no operator or contract-semantics claim.skills/**) ⇒ this PR stays DRAFT. Review requests are the seat's step; it is not flipped ready and not enqueued.content/docs/releases/untouched. Assignee and card labels untouched.🤖 Generated with Claude Code
https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
Generated by Claude Code