Uh oh!
There was an error while loading. Please reload this page.
docs(audits): measure zod to JSON Schema conversion fidelity for @object-ui/types - #6203
Merged
yinlianghui merged 3 commits intoAug 25, 2026
Merged
Conversation
…ect-ui/types The measurement objectui#5392's 2026-08-22 ruling required before the A/B/C fork is decided. No generator, no drift gate, no published artifact. Converter: zod 4.4.3's built-in z.toJSONSchema (already a dependency). Population: AnyComponentSchema — 14 union members, 106 leaf node schemas. Headline: the generated artifact is a strict relaxation of the zod source. Over 475 root-type JSON files it agrees with safeValidateSchema on 461 (97.1%), is weaker on 14, and is stricter on none. 0/106 node types convert under default options; 106/106 convert lenient; 0/106 faithful. 285 of 342 objects are open because BaseSchema is .passthrough() and .strict() is called zero times, so the artifact reads closed and is not. Size: 666,321 bytes pretty / 376,928 minified, or +21.8% / +12.3% of the published dist. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSoz9uGhaaSgiq3hshtN7L
…n-schema-fidelity-measurement
…base The measurement objectui#5392's 2026-08-22 ruling required before the A/B/C fork is decided. No generator, no drift gate, no published artifact. Re-run on origin/main @ 50d0d6c, which contains objectui#6117 (CRUDSchema retired). Converter: zod 4.4.3's built-in z.toJSONSchema (already a dependency). Population: AnyComponentSchema — 14 union members, 105 leaf node schemas. Headline: the generated artifact is a strict relaxation of the zod source. Over 475 root-type JSON files it agrees with safeValidateSchema on 461 (97.1%), is weaker on 14, and is stricter on none. 0/105 node types convert under default options; 105/105 convert lenient; 0/105 faithful. 276 of 333 objects are open because BaseSchema is .passthrough() and .strict() is called zero times, so the artifact reads closed and is not. Size: 651,908 bytes pretty / 369,028 minified, or +21.5% / +12.2% of the published dist. New since the pre-#6117 run: all 5 refinement sites isolated with passing controls (was 2 of 5); @objectstack/spec already ships generated JSON Schema (23.4 MB, 156 UI documents, 118/124 closed at the root) as both a precedent and the closedness contrast; the JSONC-only files were checked and carry no root type, so the 475 census is complete against check.ts's own reader. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSoz9uGhaaSgiq3hshtN7L
yinlianghui
marked this pull request as ready for review
August 25, 2026 01:36
Uh oh!
There was an error while loading. Please reload this page.
yinlianghui
deleted the
claude/issue-5392-json-schema-fidelity-measurement
branch
August 25, 2026 01:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #5392
Report-only. This PR adds one document under
docs/audits/and changes nothingexecutable: no generator, no drift gate, no published artifact, no edit to
packages/types.The 2026-08-22 ruling on #5392 ordered a measurement before the A / B / C fork is
decided. ⛔ This PR does not decide it — it reports what the numbers say.
Measured on
origin/main@50d0d6cc5, merged into this branch as12dcdc110; the report commitis
b149c2b14. This base contains#6117 (105f3c55c,CRUDSchemaretired), so thecensus excludes that node type. An earlier run of this measurement was taken before that
landed; it is superseded and the delta is stated in the report.
Answers
z.toJSONSchema(), built in to zod 4.4.3 — already a dependencyAnyComponentSchema— 14 union members, 105 leaf node schemasdist.strict()appears 0 times in the mirrorsThe decisive result.
BaseSchemais.passthrough(), so every one of the 105 nodetypes projects
additionalPropertiesto an accept-anything schema. A misspelled real key(
classNmaeforclassName) is accepted by the artifact exactly as it is by zod today.The artifact would carry
properties,requiredand aconston everytype— itreads closed and is not. The card's own hypothesis, measured rather than predicted.
The contrast that proves the cause is the source, not the converter.
@objectstack/spec— already a dependency — ships generated JSON Schema today(
json-schema/, 23.4 MB, 156 UI documents). Its sources arestrictObject(...), and118 of its 124 object-rooted UI documents close at the root. Same class of pipeline,
opposite closedness.
Which fork the measurement points to
B, with C untouched by these numbers and A blocked on #5155 rather than
refuted forever.
safeValidateSchemaalready delivers everything the artifact would anddelivers it better: 97.1 % agreement over the corpus, never worse, better on 14 files.
C's value is completion, not rejection, and every degradation measured here leaves
suggestions intact. ⛔ The choice remains the maintainer's — see the report's four-way
write-up.
Verification
Numbers, not behaviour, so verification is that they are reproducible and non-vacuous.
Every count in the report carries the command that produced it, and every zero sits beside
a non-zero from the identical instrument (
z.bigint(= 0 next toz.string(= 689;{type:'module'}rejected next to{type:'button'}accepted;z.strictObjectprojectingadditionalProperties: falsein the same run in which no ObjectUI node does). All fiverefinement sites were isolated with controls that pass — none inferred. The report has its
own "what this measurement cannot see" section.
⛔ No ablation: there is no behaviour to ablate on a report-only diff. The measurement
scripts were written outside the repository and are deliberately not committed —
committing one would be the generator this card is not ruled to build.
git statusisclean apart from the report.
Gates
Report-only diff, so most of the farm has nothing to judge. ⛔ No green is claimed from a
gate that did not look.
scripts/check-changeset-presence.mjs✅ No source of a released package changed in this range, so no changeset is owed.scripts/check-control-bytes.mjs✅ check-control-bytes: OK (scanned 5122 tracked text file(s); skipped 85 binary).pnpm --filter @object-ui/types builddist/*.d.tswould corrupt the measurement in both directions, not because this diff needs itNo changeset: the gate above states none is owed. This repo has no
skip-changesetlabel, and none was created.
Out of scope, already filed
#6124 — 28 zod-mirror keys declared
z.function(), which no JSON document can satisfy;14 in-repo corpus files author them and are invalid today while sitting outside
objectui check's judgement surface. Filed unassigned, ⛔ not touched here.Generated by Claude Code