Skip to content

fix(core): rate-limit ExpressionEvaluator's fault warnings to one per authored source - #6492

Merged
os-support-ai merged 2 commits into
mainfrom
claude/issue-6444-evaluator-warn-dedupe
Aug 26, 2026
Merged

fix(core): rate-limit ExpressionEvaluator's fault warnings to one per authored source#6492
os-support-ai merged 2 commits into
mainfrom
claude/issue-6444-evaluator-warn-dedupe

Conversation

@os-support-ai

@os-support-aios-support-ai commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Fixes#6444

ExpressionEvaluator.evaluate()'s two built-in fault paths logged on every evaluation, and it is the hottest of the three predicate paths in this area — SchemaRenderer calls it for every properties.* value, every props.* value and content, for every node, on every render. This gives them the same one-per-source rate limit both sibling reporters already carry. No third mechanism is introduced, and no symbol is added to the published surface.

The flood, reproduced before the fix

Measured on the built evaluator (packages/core/dist), same script both sides, base 830ed5803:

casebeforeafter
evaluateCondition('${nosuchroot.x > 1}') x3 (the card's headline)3 lines1
{ dialect: 'cel' } envelope x3 — the control the card compares against11
one broken ${…} prop rendered across a 200-row list200 lines1
multi-part template, per-part fault, x33 lines1

The two message texts are byte-identical before and after (Expression evaluation failed for: … / Failed to evaluate expression: …). The four cases run in one process against four distinct sources and produce four lines in total after the fix — i.e. the same run also shows the "different sources still log" direction.

⛔ Not fixed by deleting the warn: a distinct broken source still gets its own loud line; only the repeats are dropped.

The card's open point: keyed on source text alone, and why

Chosen: the predicate's authoring identity — [site, source] — never the scope it ran against.

Read from the sibling reporters first, as triage directed. Both key on authoring identity and neither keys on the data:

  • fieldRules.tswarnPredicateFailureJSON.stringify([expr.dialect, expr.source]), because "a broken predicate is re-evaluated on every render/keystroke, and the point is one loud line, not a scrolling wall".
  • packages/react/src/utils/visibilityDiagnostic.tsJSON.stringify([type, key, predicateSourceText(raw)]), where type/key are the authoring location (node type, prop key), not the row: "the same broken predicate authored once and rendered over many rows is ONE authoring bug, and an object key would report it once per row".

Here the precedent is also the defect. The 200-row flood is one authored source evaluated against 200 different scopes, so a scope-sensitive key is not a weaker fix — it is no fix. Replayed over the same real fault stream:

raw fault events (today: one console line each): 200
dedupe keyed on SOURCE TEXT ALONE -> lines: 1
dedupe keyed on SOURCE + SCOPE -> lines: 200

The site half of the key ('template-part' vs 'whole-expression') is not a scope discriminator — it is the same defensive tagging visibilityDiagnostic gives its two legs so two different faults cannot silence each other. The card's counter-consideration (the same text genuinely broken in one scope and fine in another) is real but is the caller's to report: onFault is deliberately left outside the rate limit, so a caller doing per-node reporting keeps every fault.

What deliberately does not move

Clause ②: the published surface is unchanged — chain followed, not grepped

packages/core is published and the chain is a two-hop wildcard, so a name grep on the entry would prove nothing:

packages/core/src/evaluator/ExpressionEvaluator.ts
-> packages/core/src/evaluator/index.ts:10 export * from './ExpressionEvaluator.js';
-> packages/core/src/index.ts:44 export * from './evaluator/index.js';

Anything exported from the source file would land on the published surface. Nothing is: EvaluationFaultSite, warnedEvaluationFaults and reportEvaluationFault are all module-local. Measured rather than asserted, before/after a rebuild:

  • packages/core/dist/**/*.d.ts93 files, all 93 SHA-256 hashes identical (diff of the hash lists is empty).
  • Runtime entry — Object.keys(await import('@object-ui/core')) is 283 names before, 283 after, diff empty.

No test-only reset export was added either. This follows the in-package precedent (fieldRules.ts's dedupe exports no reset); @object-ui/react's __resetVisibilityPredicateWarnings was not copied because no consumer needs it — the tests here use the sanctioned vi.resetModules() path instead.

Tests — both directions, plus the cell that discriminates the granularity

New: packages/core/src/evaluator/__tests__/ExpressionEvaluator.faultWarnDedupe.test.ts (8 cells).

Either direction alone is vacuous — "same source logs once" passes if everything was silenced, "different sources log twice" passes if nothing was deduped — so both are pinned, together with the cell that separates the two candidate keyings (one source, 200 different scopes, one line: 1 under the shipped keying, 200 under source+scope).

Anti-ghost-assertion: every dedupe cell also asserts the fault really happened (the documented fail-soft value came back) and that a healthy expression in the same run still evaluates to its real value — so "one warn" cannot be an evaluator that stopped being called. The 200-row cell asserts all 200 renders produced 200 distinct outputs interpolating each row's own id.

Cross-test leakage was the live hazard here (a module-level Set outlives a test case). beforeEach does vi.resetModules() + a fresh dynamic import — the exemption object-ui/no-dynamic-import-in-test-hook documents for exactly this — cells use distinct source texts anyway, and one cell proves the reset works rather than assuming it: same source deduped within one module instance, then vi.resetModules(), then the same source warns again.

Updated: ExpressionEvaluator.onFault.test.ts. Its last cell asserted "nothing about the existing console output moves" and reused FAULT_TEMPLATE, which an earlier cell in the same file already faults on — after this change it would have read that cell's dedupe entry, seen silence, and passed having measured nothing. It now faults on a source unique to the file, and its title says what it actually pins.

Ablation — the right cells red, the controls green

Rate limit removed (message text left untouched), mutation confirmed on disk in both directions (injected marker 1, removed guard 0; blob 678d9b85 -> 78fb1620), then restored with a path-scoped git checkout HEAD against the file's absolute path under a trap … EXIT INT TERM, and verified byte-identical (git hash-object back to 678d9b85, git diff HEAD empty):

Tests 3 failed | 13 passed (16)
x SAME source, three evaluations: ONE line -> expected 1 times, but got 3 times
x THE OPEN POINT: 200 DIFFERENT scopes -> expected 1 times, but got 200 times
x the dedupe survives new evaluator INSTANCES -> expected 1 times, but got 2 times
✓ DIFFERENT sources: each still gets its own line (control)
✓ the two fault SITES report independently (control)
✓ a HEALTHY expression never warns (control)
✓ `onFault` still fires on EVERY fault (control)
✓ `throwOnError` still throws on EVERY evaluation (control)
✓ all 8 cells of ExpressionEvaluator.onFault.test.ts (control)

The ablated numbers reproduce the card's measurements exactly (3 and 200). The mutation was made to source and the tests import ../ExpressionEvaluator.js relatively, so this run is also the proof that these cells read source rather than a stale dist/.

Gates run locally — union re-run on the final commit 4b1196142

gateits own verdict line
vitest run packages/core/Test Files 102 passed (102) / Tests 2039 passed (2039)
vitest run packages/react/ packages/components/ packages/plugin-detail/Test Files 360 passed (360) / Tests 3620 passed (3620)
turbo run type-check --filter=@object-ui/coreTasks: 3 successful, 3 total
turbo run build --filter=@object-ui/coreTasks: 2 successful, 2 total
eslint . in packages/core✖ 515 problems (0 errors, 515 warnings), exit 0
check-control-bytes✅ check-control-bytes: OK (scanned 5369 tracked text file(s); skipped 85 binary)
check-changeset-presence✅ 3 source file(s) of 1 released package(s) changed, and this change declares 1 changeset(s)
check-changeset-no-major✅ No changeset declares a 'major' bump.
check-vi-mock-specifiers✅ check-vi-mock-specifiers: OK
check-package-self-import✅ No package names itself inside its own src/.

Exit codes were captured before any pipe (redirect to a file, then read $?), never from a tail.

packages/core's type-check is tsc --noEmit && tsc -p tsconfig.test.json, and --listFiles confirms both edited test files are in the checked set — so "typecheck green" is a real statement about the new tests, not a silently-excluded one.

Lint narrowing, measured.pnpm lint is turbo run lint (per-package eslint .); only packages/core was run. (1) The population comes from eslint's own config, not my guess: (2) --format json reports 197 files selected under packages/core. (3) eslint.config.js sets no project/projectService, i.e. type-aware linting is not enabled, so a file's verdict depends only on its own text plus config — a diff confined to packages/core/src/evaluator/ cannot move the verdict on any untouched file in any other package, and .changeset/*.md is outside the config's **/*.{ts,tsx} selector entirely.

Not measured, reported as such:check-readme-exports exits 1 in this worktree with 356 self-import(s) could not be judged … type entry ./dist/index.d.ts is not on disk — run 'pnpm build' first. That is an unbuilt-worktree prerequisite, not a finding: zero of its 356 items concern packages/core or any file in this diff, and this PR touches no README and adds no export. CI builds first and runs it properly.

.changeset/6444-evaluator-fault-warn-dedupe.mdpatch on @object-ui/core (never major; fixed group).

Draft on purpose, no auto-merge — the PM lands it.


Generated by Claude Code

… source
`evaluate()`'s two built-in fault paths logged on EVERY evaluation, and it is
the hottest of the three predicate paths in this area — `SchemaRenderer` calls
it for every `properties.*` value, every `props.*` value and `content`, for
every node, on every render. Measured on the built evaluator at 830ed58:
three identical faulting `evaluateCondition` calls produced 3 console lines
where the `{ dialect: 'cel' }` envelope produced 1, and one broken `${…}` prop
across a 200-row list produced 200 lines per render.
Both sibling reporters already carry a one-per-source rate limit
(`warnPredicateFailure` in `fieldRules.ts`, `visibilityDiagnostic.ts` in
`@object-ui/react`); this reuses that shape rather than adding a third. The key
is the authoring identity — `[site, source]`, never the scope — which is both
the siblings' precedent and the defect itself: the 200-row flood is ONE
authored source against 200 distinct scopes, so a scope-sensitive key emits all
200 lines again.
The rate limit governs the built-in line only: `onFault` still fires on every
fault (#6038's passback contract) and `throwOnError` still throws every time.
No symbol is added to the published surface — the whole `packages/core`
`.d.ts` tree is byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011SfZeFWrhGLHmfq61xbz4q
@github-actions

Copy link
Copy Markdown
Contributor

✅ Console Performance Budget

MetricValueBudget
Eager closure (gzip, 52 chunks)3234.5 KB3266.6 KB
Main entry chunk (gzip)157.4 KB350 KB
Entry fileindex-BlUutpwf.js
StatusPASS

The eager closure is every chunk the entry reaches through static imports — what the browser fetches and parses before the app renders. The entry chunk on its own is a small fraction of it.


📦 Bundle Size Report

PackageSizeGzipped
app-shell (consoleActionDispatch.js)0.20KB0.19KB
app-shell (index.js)11.30KB4.28KB
app-shell (runtime-config.js)18.10KB6.51KB
app-shell (types.js)0.01KB0.04KB
app-shell (urlParams.js)10.06KB3.86KB
auth (ActiveOrganizationStorage.js)25.05KB9.16KB
auth (AuthContext.js)0.31KB0.24KB
auth (AuthGuard.js)2.07KB1.00KB
auth (AuthProvider.js)40.18KB10.59KB
auth (AuthShell.js)3.49KB1.40KB
auth (ForgotPasswordForm.js)12.21KB3.45KB
auth (LoginForm.js)18.15KB5.39KB
auth (PreviewBanner.js)0.90KB0.50KB
auth (RegisterForm.js)6.65KB2.22KB
auth (SocialSignInButtons.js)9.61KB3.89KB
auth (UserMenu.js)3.41KB1.23KB
auth (auth-gate-events.js)1.29KB0.66KB
auth (authStyles.js)5.04KB1.72KB
auth (createAuthClient.js)40.21KB10.80KB
auth (createAuthenticatedFetch.js)8.46KB3.43KB
auth (index.js)3.19KB1.44KB
auth (invitation-status.js)1.22KB0.70KB
auth (org-roles.js)6.66KB2.78KB
auth (phone-identifier.js)1.11KB0.66KB
auth (types.js)0.59KB0.35KB
auth (useAuth.js)5.30KB1.02KB
auth (useWorkspaceAdminStatus.js)5.13KB2.35KB
collaboration (CommentThread.js)26.08KB7.56KB
collaboration (LiveCursors.js)3.17KB1.27KB
collaboration (PresenceAvatars.js)6.49KB2.64KB
collaboration (PresenceProvider.js)2.79KB1.13KB
collaboration (index.js)1.68KB0.73KB
collaboration (useCollaborationTranslation.js)6.05KB2.52KB
collaboration (useCommentSearch.js)1.98KB0.88KB
collaboration (useConflictResolution.js)7.75KB1.86KB
collaboration (useMentionNotifications.js)1.81KB0.68KB
collaboration (usePresence.js)6.33KB1.84KB
collaboration (useRealtimeSubscription.js)7.91KB2.01KB
components (index.js)505.99KB114.64KB
core (index.js)5.30KB2.13KB
create-plugin (index.js)10.08KB3.26KB
data-objectstack (index.js)173.10KB47.96KB
fields (index.js)238.89KB60.02KB
i18n (LocalizationContext.js)1.76KB0.96KB
i18n (currency.js)1.22KB0.64KB
i18n (fallbackInterpolation.js)6.25KB2.77KB
i18n (i18n.js)4.28KB1.75KB
i18n (index.js)3.44KB1.39KB
i18n (pickLocalized.js)7.62KB3.26KB
i18n (provider.js)26.89KB9.04KB
i18n (useDisplayLocale.js)2.85KB1.45KB
i18n (useObjectLabel.js)33.40KB8.71KB
i18n (useSafeTranslation.js)5.60KB2.33KB
layout (index.js)38.95KB10.97KB
mobile (MobileProvider.js)0.92KB0.49KB
mobile (ResponsiveContainer.js)0.94KB0.38KB
mobile (breakpoints.js)1.51KB0.70KB
mobile (createOfflineDataSource.js)5.61KB1.75KB
mobile (index.js)1.55KB0.62KB
mobile (offlineQueue.js)3.91KB1.35KB
mobile (pwa.js)0.97KB0.49KB
mobile (serviceWorker.js)1.48KB0.62KB
mobile (serviceWorkerSource.js)3.41KB1.48KB
mobile (useBreakpoint.js)1.54KB0.65KB
mobile (useGesture.js)6.96KB1.98KB
mobile (useOfflineSync.js)1.99KB0.72KB
mobile (usePullToRefresh.js)2.53KB0.85KB
mobile (useResponsive.js)0.72KB0.42KB
mobile (useResponsiveConfig.js)1.37KB0.63KB
mobile (useSpecGesture.js)4.32KB1.64KB
mobile (useTouchTarget.js)1.01KB0.54KB
permissions (MePermissionsProvider.js)9.53KB3.38KB
permissions (PermissionContext.js)0.31KB0.25KB
permissions (PermissionGuard.js)0.89KB0.45KB
permissions (PermissionProvider.js)4.64KB1.50KB
permissions (evaluator.js)5.12KB1.74KB
permissions (index.js)0.93KB0.41KB
permissions (store.js)0.91KB0.42KB
permissions (useFieldPermissions.js)1.28KB0.53KB
permissions (usePermissions.js)1.93KB0.88KB
plugin-ai (index.js)15.75KB3.80KB
plugin-calendar (index.js)46.91KB12.92KB
plugin-charts (index.js)64.66KB18.32KB
plugin-chatbot (index.js)188.60KB44.82KB
plugin-dashboard (index.js)133.48KB34.49KB
plugin-designer (index.js)211.90KB42.74KB
plugin-detail (index.js)245.29KB62.39KB
plugin-editor (index.js)2.46KB1.10KB
plugin-form (index.js)131.78KB32.19KB
plugin-gantt (index.js)164.14KB39.87KB
plugin-grid (index.js)201.66KB54.57KB
plugin-kanban (index.js)53.16KB14.65KB
plugin-list (index.js)112.74KB27.50KB
plugin-map (index.js)20.09KB6.62KB
plugin-markdown (index.js)13.72KB4.69KB
plugin-report (index.js)43.51KB11.94KB
plugin-timeline (index.js)26.72KB7.71KB
plugin-tree (index.js)9.26KB3.13KB
plugin-view (index.js)84.85KB20.79KB
providers (DataSourceProvider.js)0.75KB0.39KB
providers (MetadataProvider.js)1.37KB0.59KB
providers (ThemeProvider.js)1.90KB0.85KB
providers (UploadProvider.js)11.66KB3.50KB
providers (index.js)0.45KB0.23KB
providers (types.js)0.01KB0.04KB
react-runtime (index.js)5.62KB2.34KB
react (LazyPluginLoader.js)4.47KB1.63KB
react (SchemaRenderer.js)56.69KB19.03KB
react (data-invalidation.js)5.05KB2.08KB
react (index.js)2.05KB1.04KB
react (schema-input.js)2.32KB1.24KB
react (spec-input.js)0.20KB0.18KB
sdui-parser (codegen.js)5.41KB2.34KB
sdui-parser (dashboard-widget-options.js)3.08KB1.30KB
sdui-parser (index.js)4.93KB2.24KB
sdui-parser (input-type.js)2.84KB1.40KB
sdui-parser (parse.js)12.13KB3.65KB
sdui-parser (provenance.js)3.66KB1.82KB
sdui-parser (types.js)0.28KB0.23KB
sdui-parser (validate.js)7.54KB2.63KB
types (ai.js)0.20KB0.17KB
types (api-types.js)0.20KB0.18KB
types (app.js)2.87KB0.99KB
types (base.js)0.20KB0.18KB
types (blocks.js)0.20KB0.18KB
types (complex.js)2.74KB1.41KB
types (crud.js)0.20KB0.18KB
types (dashboard-filter-alias.js)6.23KB2.74KB
types (data-display.js)3.75KB1.85KB
types (data-protocol.js)0.20KB0.19KB
types (data.js)0.20KB0.18KB
types (designer.js)1.85KB0.85KB
types (disclosure.js)0.20KB0.18KB
types (error-code.js)1.54KB0.88KB
types (feedback.js)0.20KB0.18KB
types (field-types.js)0.20KB0.18KB
types (form.js)0.20KB0.18KB
types (http-inflight.js)8.87KB3.73KB
types (http-retry.js)4.32KB2.02KB
types (icon-key-migration.js)4.26KB1.63KB
types (index.js)4.72KB2.24KB
types (layout.js)0.20KB0.18KB
types (managed-by.js)0.19KB0.18KB
types (mobile.js)2.59KB1.31KB
types (navigation.js)0.20KB0.18KB
types (objectql.js)0.20KB0.18KB
types (overlay.js)0.20KB0.18KB
types (permissions.js)0.20KB0.18KB
types (plugin-scope.js)0.20KB0.18KB
types (record-components.js)0.20KB0.19KB
types (record-semantics.js)1.28KB0.67KB
types (registry.js)0.20KB0.18KB
types (reports.js)0.20KB0.18KB
types (spec-report.js)5.05KB1.93KB
types (spec-ui-namespace.js)0.20KB0.19KB
types (system-fields.js)3.33KB1.54KB
types (theme.js)6.28KB2.87KB
types (ui-action.js)3.40KB1.71KB
types (views.js)0.20KB0.18KB
types (widget.js)0.20KB0.18KB

Size Limits

  • ✅ Core packages should be < 50KB gzipped
  • ✅ Component packages should be < 100KB gzipped
  • ⚠️ Plugin packages should be < 150KB gzipped

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ExpressionEvaluator.evaluate warns ONCE PER EVALUATION on a faulting ${…} — a broken prop in a 200-row list is 200 console lines per render

2 participants

@os-support-ai@claude