Skip to content

docs: finish implementation-accuracy audit — durable reference grouping + hand-written doc sync (#1880) - #3243

Merged
os-zhuang merged 11 commits into
mainfrom
claude/implementation-accuracy-audit-eavdgx
Jul 19, 2026
Merged

docs: finish implementation-accuracy audit — durable reference grouping + hand-written doc sync (#1880)#3243
os-zhuang merged 11 commits into
mainfrom
claude/implementation-accuracy-audit-eavdgx

Conversation

@os-zhuang

@os-zhuangos-zhuang commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Continues the docs implementation-accuracy effort (#1880; follows #1866 / #1904). Wave 1 audit complete — 164/164 hand-written docs.

Summary

  • Wave 1 — hand-written doc audit: 164/164 docs via a multi-agent pipeline (per doc: locate the implementation in packages/ → surgical fix → adversarial verify), all in a dedicated worktree.
    • 335 evidence-backed fixes across 116 docs (the other 48 were already accurate → no change).
    • 14 of those were verifier repairs — the adversarial pass catching and correcting a wrong audit edit before it landed (e.g. a fabricated permissionSet: stack key → the real permissions:; a fabricated projection claim removed).
    • 259 report-only residuals logged — ambiguous / aspirational / frontmatter-locked items the agents correctly did not "fix".
  • Wave 2.1 — durable reference grouping: moved docs(nav): group Protocol Reference modules and clarify top-level nav #1862's per-category sidebar grouping into build-docs.ts (fumadocs ---Section--- separators, drift-safe). gen:docs --check green.
  • Wave 2.2 — generated-reference quality: generator formatting verified solid (no dangling links, valid tables); filled ui/view describe strings. The ~1,180-property describe backlog (mostly self-documenting z.literal discriminants) is tracked continuation.

Representative fixes: removed the retired feed client namespace; password fields are masked-on-read, not hashed; the certified measure flag is ADR-0021-not-yet-implemented; removed softDelete / versioning / fileAttachmentConfig / vectorConfig (all removed in 16.x, #2377); referenceFilterslookupFilters; fabricated definePlugin() / defineObject()ObjectSchema.create; error-handling fabricated SCREAMING_SNAKE codes + WebSocket/Business-Logic sections → real StandardErrorCode values.

Method / quality notes

  • Every fix cites file:line evidence; unverifiable claims were left unchanged per rule (e.g. I reverted one percent 0-1↔0-100 change I couldn't confirm in this repo — the scale lives in the objectui renderer).
  • A few batches had their adversarial-verify phase cut short by usage limits; those docs were audit-only and hand-checked against the spec before commit. The two protocol docs whose first audit was interrupted mid-edit were reverted and cleanly re-audited.
  • Vercel docs build (gen:schema && gen:docs && next build) green on every push.

Flagged follow-ups from the issue — both resolved

CODE / frontmatter follow-ups surfaced by the audit (out of scope for this docs-only PR)

Real implementation gaps the audit found but couldn't fix here:

  • explain: os explain object documents ownership as "own" | "extend" — real values are user | org | none #3244os explain object documents ownership as "own" | "extend" (real: user | org | none). (filed)
  • kernel/contracts/cache-service.mdxfrontmatter claims "namespaces, and bulk operations" the contract doesn't have (preserve-frontmatter rule blocked the body-only fix).
  • OS_DATABASE_DRIVER=memory has no branch in serve.ts dispatch (falls through to SQLite in-memory).
  • @objectstack/driver-sql declares peerDep sqlite3 but loads better-sqlite3 at runtime.
  • formula js dialect is registered only as a stub (hasDialect('js') is false).
  • protocol/knowledge.mdx has broken ai-capabilities.mdx cross-references (target removed in the reorg).
  • glossary "TCK (Technology Compatibility Kit)" is grep-empty (aspirational term).

Wave 2.2 describe-string backlog (continuation of #1880)

~1,180 generated-reference properties still lack descriptions — most are self-documenting z.literal discriminants; the genuine gaps concentrate in internal protocol schemas (e.g. data-engine). Left for a follow-up.


Left as a draft for maintainer review — not auto-merging.

🤖 Generated with Claude Code

https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt

claude added 3 commits July 18, 2026 15:31
gen:docs rewrote each references/{cat}/meta.json as a flat alphabetical
list on every run, silently discarding the per-category grouping that was
added by hand in #1862. Move the grouping into build-docs.ts so it
survives regeneration: a SECTION_GROUPS map drives fumadocs "---Section---"
separators for the 9 large categories (ai, api, automation, cloud, data,
integration, kernel, system, ui); smaller categories keep the flat list.
The mapping degrades gracefully — pages present on disk but unmapped still
appear (under a trailing "More" separator) and names that produce no page
are ignored — so adding or removing a schema never drops it from the
sidebar. `gen:docs --check` stays green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…iew config (#1880)
Fill empty zod .describe() strings on the most author-facing view schema
so the generated ui/view reference prose is useful:
- CalendarConfig / GanttConfig field references (startDateField,
endDateField, titleField, colorField, progressField, dependenciesField)
- the type-specific view config objects (kanban / calendar / gantt / tree)
Regenerated content/docs/references/ui/view.mdx from the updated schema.
Part of the Wave 2.2 describe-string effort; the broader backlog
(~1,180 empty descriptions, concentrated in internal protocol schemas
such as data-engine) remains as tracked continuation of #1880.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…Wave 1, batch 1)
First batch of evidence-backed accuracy fixes from the multi-agent doc
audit (one agent per doc → locate implementation in packages/ →
adversarial verify). The api section (11 docs) is fully audited and
adversarially verified — 12 fixes, 0 verifier repairs; the ai/automation
docs here are audit-verified.
Representative fixes: removed the retired `feed` client namespace,
corrected the connect discovery response shape (features → capabilities),
and fixed the aiService.chatWithTools() call signature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
@vercel

vercelBot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
specReadyReadyPreview, CommentJul 19, 2026 12:53pm

Request Review

…batch 2)
More evidence-backed accuracy fixes from the audit workflow. The adversarial
verify phase was cut short by a session usage limit, so each fix here was
additionally hand-checked against the spec before committing:
- data-modeling/field-type-decision-tree: `password` is masked-on-read, not
hashed — prefer `secret` for real credentials (#2036).
- data-modeling/analytics: the `certified` measure flag is an ADR-0021 design
goal, not yet implemented (DatasetMeasureSchema has no such field) — removed
the example usage and corrected the guidance.
- concepts/metadata-driven: config-key example `referenceFilters` →
`lookupFilters` (referenceFilters removed in 16.x, #2377 / ADR-0049).
- concepts/metadata-lifecycle: per-org overlay table listed removed
`role`/`profile` types → `permission`/`position`.
- concepts/index: dropped the auto-GraphQL claim (REST is auto-generated).
- automation/webhooks: sys_webhook filter field `events[]` → `triggers`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…lementation (#1880 Wave 1, batch 3)
30 docs audited AND adversarially verified this window; 20 carry
evidence-backed fixes. Highlights:
- field-types: `password` is masked-on-read (ADR-0100), not hashed — real
credential hashing is owned by better-auth (sys_account.password); removed
fabricated `fileAttachmentConfig` and `vectorConfig` (both removed in the
16.x line, #2377).
- objects: removed non-existent `tags` / `recordName` props and the
`softDelete` / `versioning` config blocks (removed in 16.0, #2377 / ADR-0049,
now tombstoned); corrected `feeds` / `activities` capability defaults (both
default `true`).
- validation-rules, getting-started/cli, deployment/*, and others: assorted
API name, flag, path and example corrections.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…ave 1, batch 4)
30 docs audited AND adversarially verified; 18 carry evidence-backed fixes
(getting-started remainder + kernel contracts / runtime-services). Highlights:
- kernel/contracts/data-engine: `@objectstack/spec` → `@objectstack/spec/data`
subpath import; trimmed the error-code table to codes the engine actually
emits (with HTTP mappings); removed the unsupported `top` → `limit` OData alias.
- kernel/events, cluster, contracts/*, runtime-services/*: assorted API name,
type, and import-path corrections.
- getting-started: AI-primitives scope note (NLQ / RAG-orchestration /
predictive pipelines were removed in v1); quick-reference and
your-first-project example fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…1880 Wave 1, batch 5)
Commits the 23 permissions/plugins/protocol docs whose audit completed
cleanly this window. Highlights:
- plugins/*: removed the fabricated `definePlugin()` helper from examples —
a plugin is a plain default-export object; no such helper exists.
- permissions/explain: corrected the explain `via` value to "direct grant"
(adversarial-verify repair); permission-metadata, administrator-guide,
authentication, field-level-security, sharing-rules: scope/flag/API fixes.
- protocol/diagram, protocol/kernel/config-resolution: path/reference fixes.
Some docs' verify pass was cut short by a weekly usage limit, so a subset is
audit-only (hand-scanned: no define*/scale/count regressions; MDX build-safe).
Two docs whose audit itself was interrupted (protocol/kernel/error-handling
and http-protocol) are reverted and deferred for a clean re-audit after the
limit resets.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…ave 1, batch 6)
30 docs audited AND adversarially verified (protocol, releases, ui); 28 carry
fixes — 103 fixes total, 8 of them verifier repairs (the adversarial pass
caught and corrected wrong audit edits, e.g. a bad `permissionSet:` stack key
→ `permissions:`, and a fabricated projection "id/audit columns" claim).
Highlights:
- protocol/kernel/*: removed the fabricated `definePlugin()` / `defineObject()`
helpers (use ObjectSchema.create) and corrected `defineView` imports to
`@objectstack/spec/ui`; aspirational manifest/lifecycle blocks left in place
where the page already discloses them as design-intent.
- protocol/knowledge: driver-turso / knowledge-turso are cloud-only (not
open-core) → replaced with open-core-appropriate driver-mongodb / llamaindex.
- releases/v13,v15 + objectql/* + ui/*: import-path, key-name, and example fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…-protocol (#1880 Wave 1, batch 7 — final)
Final batch — completes the 164-doc hand-written implementation-accuracy
audit. 13 docs audited AND adversarially verified. Includes a clean
re-audit of the two protocol docs whose first pass was interrupted by a
usage limit (their partial diffs were reverted earlier):
- protocol/kernel/error-handling: replaced fabricated SCREAMING_SNAKE error
codes with the real lowercase StandardErrorCode values (unauthenticated,
permission_denied, resource_not_found, delete_restricted,
rate_limit_exceeded, service_unavailable, …) and removed the fabricated
"WebSocket-Specific Errors" / "Business Logic Errors" sections — the error
contract is HTTP/REST, not multi-channel.
- protocol/kernel/http-protocol: corrected discovery routes, response-shape
names, and fetch examples to the real /api/v1 surface.
- ui/{forms,views,dashboards,pages,setup-app,...}: example, flag and
field-name corrections.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
Bring the docs-audit branch up to date with main (29 commits) before merge.
Conflict resolutions:
- content/docs/automation/flows.mdx: took main's newer run-history paragraph
(sys_automation_run durable history / bounded step log) which superseded this
branch's audit wording of the same section.
- content/docs/references/automation/meta.json: regenerated; added main's new
`time-relative-trigger` schema to the automation "Flow & Execution" sidebar
grouping in build-docs.ts. `gen:docs --check` green (254 files in sync).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation protocol:ui tooling labels Jul 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

103 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx(via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx(via @objectstack/spec)
  • content/docs/ai/skills.mdx(via @objectstack/spec)
  • content/docs/api/client-sdk.mdx(via @objectstack/spec)
  • content/docs/api/environment-routing.mdx(via @objectstack/spec)
  • content/docs/api/error-catalog.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx(via @objectstack/spec)
  • content/docs/api/index.mdx(via @objectstack/spec)
  • content/docs/automation/approvals.mdx(via packages/spec)
  • content/docs/automation/flows.mdx(via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx(via packages/spec)
  • content/docs/automation/hooks.mdx(via @objectstack/spec)
  • content/docs/automation/index.mdx(via @objectstack/spec)
  • content/docs/automation/webhooks.mdx(via @objectstack/spec)
  • content/docs/automation/workflows.mdx(via @objectstack/spec)
  • content/docs/concepts/architecture.mdx(via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx(via packages/spec)
  • content/docs/concepts/index.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx(via packages/spec)
  • content/docs/concepts/north-star.mdx(via packages/spec)
  • content/docs/data-modeling/analytics.mdx(via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx(via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx(via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx(via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx(via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx(via @objectstack/spec)
  • content/docs/data-modeling/index.mdx(via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx(via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx(via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx(via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx(via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx(via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx(via @objectstack/spec)
  • content/docs/getting-started/cli.mdx(via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx(via @objectstack/spec)
  • content/docs/getting-started/examples.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx(via @objectstack/spec)
  • content/docs/getting-started/validating-metadata.mdx(via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx(via @objectstack/spec)
  • content/docs/kernel/cluster.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx(via packages/spec)
  • content/docs/kernel/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx(via packages/spec)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/spec)
  • content/docs/permissions/authorization.mdx(via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx(via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx(via @objectstack/spec)
  • content/docs/permissions/positions.mdx(via @objectstack/spec)
  • content/docs/permissions/rls.mdx(via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx(via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx(via @objectstack/spec)
  • content/docs/plugins/development.mdx(via @objectstack/spec)
  • content/docs/plugins/index.mdx(via @objectstack/spec)
  • content/docs/plugins/packages.mdx(via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx(via @objectstack/spec)
  • content/docs/protocol/diagram.mdx(via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx(via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx(via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx(via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx(via @objectstack/spec)
  • content/docs/releases/index.mdx(via @objectstack/spec)
  • content/docs/releases/v12.mdx(via @objectstack/spec)
  • content/docs/releases/v13.mdx(via @objectstack/spec)
  • content/docs/releases/v9.mdx(via @objectstack/spec)
  • content/docs/ui/actions.mdx(via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx(via @objectstack/spec)
  • content/docs/ui/dashboards.mdx(via @objectstack/spec)
  • content/docs/ui/forms.mdx(via @objectstack/spec)
  • content/docs/ui/index.mdx(via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx(via @objectstack/spec)
  • content/docs/ui/setup-app.mdx(via @objectstack/spec)
  • content/docs/ui/translations.mdx(via @objectstack/spec)
  • content/docs/ui/views.mdx(via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review July 19, 2026 12:46
The audit reduced discouraged "role" wording in 10 docs; commit the improved
baseline so `check:role-word` locks in the improvement (per the check's own
ratchet-down instruction).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationprotocol:uisize/xltooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@os-zhuang@claude