Uh oh!
There was an error while loading. Please reload this page.
docs(skills): objectstack-formula factual sweep — every behavioral claim verified against the implementation - #13674
Merged
Conversation
…aim verified against the implementation Six measured falsehoods corrected, token-neutral under the skills ratchet (6002/6002, no ceiling raise). Verified against `@objectstack/formula`, `@objectstack/lint` and `@objectstack/spec` by reading plus executed probes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EXxTW8mvPBhoHxmyPZ63de
This was referenced Aug 31, 2026
Merged
os-zhuang
approved these changes
Aug 31, 2026
os-zhuang
marked this pull request as ready for review
August 31, 2026 07:15
os-zhuang
enabled auto-merge
August 31, 2026 07:15
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#13660
Program anchor: #13658 — flight ① of the published-skills factual sweep, the calibration flight. That anchor stays open; this PR carries only its member ①, and the roster and re-sizing live on it.
Session, for durable attribution:
https://claude.ai/code/session_01EXxTW8mvPBhoHxmyPZ63de⛔ Governed surface — human merge, deliberately parked
skills/**is the published, customer-facing surface. This PR is a draft on purpose: auto-merge is not armed, it is not marked ready, no review is requested, and the dispatching seat will not arm it. It waits for a maintainer to merge by hand. A draft awaiting the maintainer's own merge is the correct terminal state here, not a stuck PR.Method — PR #13577 is the spec
Every behavioral claim in
skills/objectstack-formula/**(2 files, 579 lines) was inventoried FIRST, then verified in inventory order against the implementation —@objectstack/formula,@objectstack/lint,@objectstack/spec— never against another document. Behavior-bearing claims were verified by reading plus an executed probe; probes were written in the worktree, run, and deleted (they are evidence, not commits — none earned a place as a real pin, and the corpus already has its drift-guard inpackages/formula/src/skill-catalog-sync.test.ts).Non-vacuity control — a claim proven true by execution. SKILL.md's "Null + string throws" section asserts a verbatim runtime message. Probed against
celEngine.evaluate:The skill's own sentence names that same overload string verbatim, byte for byte, as what
null + 'foo'throws — VERIFIED against the line quoted above. Same run also pinned, by execution:floor(-1.2) == -2andceil(-1.2) == -1;addMonths(date('2026-01-31'), 1)clamping to Feb 28;today()anddaysFromNow(45)landing on UTC midnight; all 8 advertised cel-js built-ins resolving; all three Salesforce-flavor sources failing CEL compile; unknown functions (PRIOR(),ISBLANK(),isBlnk()) failing with the "no matching overload" type error the skill promises.The six FALSE claims, and what measured them
end - startdoes not fault, it yields adurationstored as{}."`defaultValue` (M9.9b)`defaultValue` (envelope only; bare string = literal)`Dataset.records[*]``Seed.records[*]``connector.schedule`, `etl.schedule`, `sync.schedule`— pipelines`connector.schedule`— scheduled connector sync`system/notification`— email subject + body, SMS message, push body + message (5 fields)`system/email-template`—subject,bodyHtml,bodyText(plain strings, the email pipeline renders the holes)`integration/connector/github`— titleTemplate, bodyTemplate (PR + release)node_modules/@objectstack/formula/src/stdlib.ts" / "…/src/validate.ts"distonly — there is nosrc/to read in an installed app. Its exportedCEL_STDLIB_FUNCTIONSis the canonical list, pinned by two tests"1.
end - startdoes not fault. The sentence grouped three shapes under "these fault". Measured side by side (build verdict viavalidateExpression, runtime viacelEngine.evaluate):timestamp - timestampis a valid CEL overload yielding agoogle.protobuf.Duration;coerce()(cel-engine.ts:1161) hands any non-Date object back key-by-key, so the field receives{}.int()andstring()both refuse a Duration, so the value is unusable rather than absent — which is why the "do NOT write it" advice stands while the stated reason did not. The implementation agrees the boundary is date against a number:validate.ts:116reads "Dates — ARITHMETIC against a number always nulls".2.
Field.defaultValueis not astring | Expressionslot. The table's own heading says "All of these spec fields acceptstring | Expression". Executed againstFieldSchemafrom@objectstack/spec/data:The control row is the point: a real CEL slot auto-wraps,
defaultValuedoes not.field.zod.ts:894states the three legal shapes — envelope, runtime TOKEN, or LITERAL. On a text field the bad spelling is silent (it stores the characterstoday()), which is exactly the AI-authoring trap the sweep exists to close.3.
Dataset.records[*]is the wrong subject.DatasetSchema(ui/dataset.zod.ts:248) is the analytics semantic layer —dimensions/measures— and has norecordskey at all. The seed-records surface isSeedSchema.records(data/seed.zod.ts:96,z.array(z.record(z.string(), z.unknown()))), registered as theseedmetadata kind. The rest of the skill already says "Seed" — the Boundaries table and pattern §4 both do — so the row was the outlier.4. Two of three named cron surfaces are retired. Measured by enumerating every
CronExpressionInputSchemause in the spec: six sites, none of them anetlor asyncschema.retry-policy.zod.ts:86records that "#6414 retired the whole L2 ETL layer", andconnector.zod.ts:21records thatautomation/sync.zod.tswas folded into the connector in #4738.connector.schedule— the surviving one — is described in-schema as "Cron expression for scheduled sync", so the row now names it that way.5 and 6. Two phantom template surfaces.
titleTemplate/bodyTemplatereturn zero hits acrosspackages/**/*.ts— the github-connector row names fields that do not exist anywhere in the repo. Thesystem/notificationrow names five fields whose schemas were removed:notification.zod.ts:40records thatEmailTemplateSchema,SMSTemplateSchema,PushNotificationSchemaandInAppNotificationSchema"were removed from this module too (ADR-0049 enforce-or-remove, v17 window)", left "reachable from no parent schema" by #4610. The same note records that push / in-app have no delivery implementation at all (#3197) — so that half of the row advertised a channel nothing delivers. The replacement row names what does carry holes today:EmailTemplateDefinitionSchema(system/email-template.zod.ts), whosesubject/bodyHtml/bodyTextare plain strings the email pipeline renders — deliberately spelled "plain strings", because atmplenvelope would fail theirz.string().7. The stdlib pointer names paths that never ship.
packages/formula/package.jsondeclaresfiles: ["dist", "README.md", "CHANGELOG.md"], andnpm pack --dry-run --jsonreports 0src/entries in the tarball. Bothnode_modules/@objectstack/formula/src/...paths are unreachable in an installed app. (@objectstack/specdoes publishsrc/**/*.zod.ts, so the two spec pointers in "See also" andreferences/_index.mdare correct and were left alone.) The replacement also states the sync guarantee precisely: two tests, not one —cel-engine.test.ts:500asserts every catalog entry resolves at runtime, andskill-catalog-sync.test.tsasserts this table documents every entry.Written to the token ratchet, not around it
This file sits at zero headroom, so every correction is paid for inside the same file. No ceiling was raised — that is a maintainer's call, not a dev's.
SKILL.mdlines (whole file)objectstack-formula/**package lines (all.md)SKILL.mdtokens (ceiling 6002)The ratchet's own verdict line at head
cb9a3cd44:Two nuances were not stuffed into ratcheted text and are recorded for a
content/docs/**follow-up instead: (a) the build rejects date-plus-number only when the numeric operand is statically typed — anumber-typed field staysdynand slips through to a runtime fault (measured above, row 3); (b) the null-guard publish gate covers three surfaces (object validation rules, lifecycle hookcondition, fieldrequiredWhen—validate-expressions.ts:820), while the skill names two.Gates — derived, not recalled, run locally at head
cb9a3cd44Family derived from the real diff with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack. 13 families; 12 green, 1 NOT MEASURED:check-test-completeness.mjsexits 3 = PREREQUISITE NOT MET by its own text (it grades a savedturbo run testlog and none was named): recorded as NOT MEASURED, not as a red.check:doc-formula-expressionsneeded@objectstack/formulaand@objectstack/lintbuilt first — its first run was a prerequisite miss, not a finding. Built, re-run, and it quotes itself:Plus two runs beyond the derived family, because they read this very corpus:
The 647-test run is at parity with the pre-edit baseline (same 25/647), and it contains
skill-catalog-sync.test.ts, the drift-guard that pins this table againstCEL_STDLIB_FUNCTIONS— so the stdlib edits did not break the catalog contract.check-nul-bytesclean:OK (scanned 7551 text file(s) … no raw ASCII control bytes).ESLint was not run repo-wide, and the narrowing is measured, not assumed. Three readings: (1) the population read from
eslint.config.mjsitself is**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}plus.ts/.tsx/.mts/.ctsblocks — no config block names.md; (2)--format jsonover the diff's only file returns 1 result with 0 messages and no block applied; (3) type-aware linting is not enabled anywhere —projectService/parserOptions.projectoccur 0 times in the config, which the file's own line 328 states — so this diff cannot move any verdict on an untouched file. Zero of the changed files are in that gate's population in either direction.No changeset
This PR releases nothing from any package — a pure
skills/**documentation correction, the repo's live convention for which is no changeset.skip-changesetis applied.Calibration numbers (this flight's purpose)
Full figures are in the report on #13660. Headline: 197 behavioral claims inventoried across 13 classes (of 579 lines, ~118 are non-behavioral — front-matter, positioning, the Boundaries routing table, Open questions, See also — and are out of the inventory by the anchor's rule). 6 FALSE (3.0%), 26 NOT MEASURABLE, 165 VERIFIED.
The density is not uniform, and this is the finding that should size flights ②–⑫: 4 of the 6 falsehoods are in the surfaces/mapping tables — 4/38 = 10.5% there against 2/159 = 1.3% everywhere else, an ~8× concentration. That matches PR #13577's pattern exactly (a mapping column asserting engine behavior), and it is the highest-yield place to point the remaining flights.
What is not here, on purpose
references/_index.mdfile is generator-owned (pnpm --filter @objectstack/spec gen:skill-refs) and correct as measured — its@objectstack/specpointers resolve, because that package does publishsrc/**/*.zod.ts. Not touched.previous-binding table and the bulk-write dispatch semantics, whose implementation lives inobjectql/runtime hook dispatch rather than@objectstack/formula, plus the determinism contract (needs two fullobjectstack buildruns). They are recorded on the card, not silently skipped.Generated by Claude Code
Generated by Claude Code