Skip to content

fix(core,cli): os test record action types reach the served route; a zero-match glob states its posture - #7982

Merged
hotlong merged 3 commits into
mainfrom
claude/issue-7848-os-test-adapter-basepath-and-empty-glob
Aug 12, 2026
Merged

fix(core,cli): os test record action types reach the served route; a zero-match glob states its posture#7982
hotlong merged 3 commits into
mainfrom
claude/issue-7848-os-test-adapter-basepath-and-empty-glob

Conversation

@hotlong

Copy link
Copy Markdown
Contributor

Fixes#7848

Two defects on the same surface, both measured on a booted showcase. Landing sites are the ones the PM claim comment established, not the triage comment's: item 1 is packages/core/src/qa/http-adapter.ts (the CLI only imports the adapter), item 2 is packages/cli/src/commands/test.ts.

Item 1 — 5 of the 8 declared action types could not reach a stock server

HttpTestAdapter built ${baseUrl}/api/data/:object. A stock server serves {apiPath}/data/:object with apiPath = /api/v1, so every record-shaped TestActionTypeSchema member was one version segment short — and update_record was wrong twice, issuing PUT at a route that has no PUT sibling.

The base path is derived, not re-written

Per the claim comment's instruction, the prefix is taken from where the server's own resolution takes it rather than hard-coded a second time. RestServer composes it as getApiBasePath() + crud.dataPrefix, i.e. api.apiPath ?? ${api.basePath}/${api.version} then + dataPrefix (packages/rest/src/rest-server.ts:3853, :6741). Both pieces are declared in packages/spec with defaults, and the adapter now asks those two schemas:

constapi=RestApiConfigSchema.parse({});// basePath '/api', version 'v1'constcrud=CrudEndpointsConfigSchema.parse({});// dataPrefix '/data'`${api.apiPath??`${api.basePath}/${api.version}`}${crud.dataPrefix}`

base_path_source: @objectstack/spec/apiRestApiConfigSchema + CrudEndpointsConfigSchema. No literal /api/v1/data survives in the adapter, and the unit pin asserts the adapter's URLs against that same derivation rather than against a copied string — a pin that hard-codes what it guards goes green the day the schema moves and the adapter does not.

Known limit, stated rather than papered over: this is the DEFAULT only. The adapter is handed an origin, not a deployment's config, so a host that sets api.apiPath or crud.dataPrefix is still out of reach for the record action types; api_call takes the path you give it and remains the escape hatch. Filed as a follow-up rather than widened here (see Out-of-scope findings).

Re-run action-type matrix

Booted showcase (pnpm dev -- --fresh -p 41847, seeded admin, session token), one scenario per member with no shared setup, so no member's verdict is inferred from a sibling — the "before" column is a real reverse-verification run (fix committed first, git checkout origin/main -- packages/core/src/qa/http-adapter.ts, core rebuilt, same suite, same server), not the card's text copied forward.

action typebeforeafter
create_recordHTTP Error 404: {"error":"Not found"}POST /api/v1/data/:object{ object, id, record }
read_record❌ 404GET /api/v1/data/:object/:id, asserted on record.name
update_record❌ 404, and PUT where the route is PATCHPATCH /api/v1/data/:object/:id, asserted on record.color
delete_record❌ 404DELETE /api/v1/data/:object/:id{ object, id, success }
query_records❌ 404POST /api/v1/data/:object/query{ object, records }
api_call✅ executes✅ executes (unchanged)
wait✅ executes✅ executes (unchanged)
run_scriptUnsupported action type in HttpAdapter: run_script❌ same — deliberately not implemented here (no adapter branch; already in the liveness ledger)

Before: FAILED: 6 scenarios failed. 2 passed. After: FAILED: 1 scenarios failed. 7 passed. — the one failure is run_script, refusing by name.

The repo's shipped suite (examples/app-showcase/qa/platform-smoke.test.json, all api_call) still passes against the same boot: SUCCESS: All 2 scenarios passed., exit 0.

Two smaller corrections ride with the verb fix, both needed for the round-trip: update_record peels id off the PATCH body (it addresses the record; posting it back is a column write), and record ids are percent-encoded.

Item 2 — the smaller ask, landed; the bigger one proposed below, not landed

The default exit status is unchanged: a zero-match glob still exits 0. What changes is that the posture is declared instead of accidental.

$ os test 'qa/nothing-matches-*.test.json'
Found 0 test suites.
No test files found matching: qa/nothing-matches-*.test.json
Exiting 0 — an empty match is not a failure. Pass --fail-on-empty to make it one.
$ echo $?
0
$ os test 'qa/nothing-matches-*.test.json' --fail-on-empty
Found 0 test suites.
No test files found matching: qa/nothing-matches-*.test.json
--fail-on-empty: a run that loaded no suite is a failed run.
$ echo $?
1
  • --help states the posture and names the opt-out.
  • --fail-on-empty is the opt-in strict reading.
  • Found N test suites. is emitted on every run, Found 0 test suites. included. It was previously printed only when N was positive — missing from exactly the run where a caller needs it to tell "every suite passed" from "there were no suites". The spelling now lives in one exported function so an edit to the prose has to notice it is editing a machine-readable surface.

Both exit-code arms are asserted, over a real child process (bin/run-dev.js + tsx, the pattern migrate-exit-code.e2e.test.ts uses) — process.exit inside a vitest worker is not an exit status. "Exits 0" is exactly the behaviour nobody writes a test for, which is how #7347's coverage.json blindness survived.

Proposal — should the hard fail become the DEFAULT? Measured evidence says no

The claim comment asked one question: do create-objectstack templates always ship a suite? Measured on this tree:

  • the bundled blank template (packages/create-objectstack/src/templates/blank/) ships 13 files and no qa/ directory at allfind … -name '*.test.json' returns nothing;
  • its package.json declares dev/start/build/validate/typecheck and no test script, so nothing in a scaffolded project invokes os test today;
  • the other templates (todo, compliance, content, …) are fetched from the separate objectstack-ai/templates repo and were not measured here — but they cannot rescue the default, because the bundled one is the fallback every offline/default scaffold gets.

So the premise that could have justified a hard-fail default is false: a freshly scaffolded project has zero suites, and flipping the default would make os test fail out of the box for the most common starting state. Recommendation: keep exit 0 as the default and leave --fail-on-empty as the opt-in. Recorded here so the question is closed with a measurement rather than re-opened on intuition; happy to be overruled by a reviewer who weighs the CI-rot risk higher.

Also updated, because they would otherwise lie

  • docs/qa/platform-checklist/areas/cli.jsoncli.qa-suite-execution revision 2. The item's own negative clause said a run finding the record types PASSING "means the adapter was repaired, which is a revision of this item rather than a silent green". This is that revision: the negative is inverted (a 404 is now the regression), the five-404s knownGap is replaced by the narrower surviving one (default mount only), the variants carry the real routes and response shapes, and the zero-match negative records the declared posture. pnpm check:platform-checklist green.
  • content/docs/deployment/cli.mdx — the flag, the empty-match posture, and the record-action-type mount limit.
  • packages/spec/liveness/qa.json — the tags row no longer claims os test has "exactly two flags"; its argument (nothing filters on tags) is untouched.

Verification

  • pnpm typecheck — clean.
  • pnpm check:type-check-debtnone above its recorded number. It caught a real one: the new core test's fetch mock typed input: RequestInfo, which does not resolve in a package whose tsc program has no DOM lib. Fixed in the test (input: unknown); the ledger was not raised.
  • pnpm --filter @objectstack/core test — 33 files, 786 tests.
  • pnpm --filter @objectstack/cli test — 114 files, 1253 tests.
  • pnpm check:platform-checklist, pnpm --filter @objectstack/spec check:liveness — green.
  • origin/main merged in and the targeted suites re-run after it.

Out-of-scope findings

  • HttpTestAdapter cannot reach a deployment that re-prefixes its API. The record action types resolve the DEFAULT mount; api.apiPath / crud.dataPrefix move it out from under them. The architecturally right fix is the one @objectstack/client uses — read routes.data from /discovery and fall back to the convention — which is a larger change than this card's closed scope. Recorded in the checklist item's knownGaps and will be filed as its own issue.
  • run_script remains declared with no adapter branch — untouched here by instruction, already carried in packages/spec/liveness/qa.json.

Generated by Claude Code

…ro-match glob states its posture (#7848)
Item 1 — `HttpTestAdapter` built `${baseUrl}/api/data/:object` while a stock
server serves `{apiPath}/data/:object` with `apiPath` = `/api/v1`, so all five
record-shaped `TestActionTypeSchema` members 404'd, and `update_record` issued
`PUT` where the route is `PATCH`. The prefix is now derived from the two schemas
`RestServer` itself resolves from (`RestApiConfigSchema` +
`CrudEndpointsConfigSchema.dataPrefix`) rather than written down a second time,
`update_record` PATCHes with `id` peeled off the body, and record ids are
percent-encoded.
Item 2 — a zero-match glob still exits 0 (a repo that legitimately ships no
suites must not start failing CI), but the posture is now declared: `--help`
states it, `--fail-on-empty` opts into the strict reading, and
`Found N test suites.` is emitted on every run including `Found 0 test suites.`
Both exit-code arms are asserted over a real child process; the adapter's URLs
and verbs are pinned against the spec schemas rather than a copied literal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1z19epfecapa69CqwZ3Mm
…paired adapter (#7848)
The item's own `negative` clause said a future run finding the record action
types PASSING "means the adapter was repaired, which is a revision of this item
rather than a silent green" — so this is that revision (revision 2, with its
history entry). The five-404s knownGap is replaced by the narrower one that
survives (the record types address the DEFAULT mount only, so a host that moves
it with `api.apiPath`/`crud.dataPrefix` still needs `api_call`), the variants
carry their new routes and response shapes, and the zero-match negative records
the now-declared posture plus `--fail-on-empty`.
Also: the core test's fetch mock takes `input: unknown` — this package's tsc
program has no DOM lib, so `RequestInfo` does not resolve and the debt ratchet
catches it (it compiles *.test.ts, which the package `typecheck` skips).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1z19epfecapa69CqwZ3Mm
@vercel

vercelBot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 12, 2026 9:22am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/cli, @objectstack/core, @objectstack/spec.

116 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/actions-as-tools.mdx(via @objectstack/core)
  • content/docs/ai/agents.mdx(via @objectstack/spec)
  • content/docs/ai/knowledge-rag.mdx(via @objectstack/core)
  • content/docs/ai/natural-language-queries.mdx(via @objectstack/core)
  • content/docs/ai/skills-reference.mdx(via packages/cli, @objectstack/spec)
  • content/docs/ai/skills.mdx(via @objectstack/spec)
  • content/docs/api/client-sdk.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/api/data-flow.mdx(via @objectstack/cli)
  • content/docs/api/environment-routing.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/api/error-catalog.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/api/error-handling-client.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx(via @objectstack/spec)
  • content/docs/api/index.mdx(via @objectstack/spec)
  • content/docs/automation/approvals.mdx(via @objectstack/spec)
  • content/docs/automation/connectors.mdx(via @objectstack/spec)
  • content/docs/automation/flows.mdx(via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx(via packages/cli, packages/spec)
  • content/docs/automation/hooks.mdx(via @objectstack/spec)
  • content/docs/automation/index.mdx(via @objectstack/spec)
  • content/docs/automation/webhooks.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/automation/workflows.mdx(via @objectstack/spec)
  • content/docs/concepts/architecture.mdx(via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx(via packages/spec)
  • content/docs/concepts/index.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx(via packages/spec)
  • content/docs/concepts/north-star.mdx(via packages/core, @objectstack/spec)
  • content/docs/data-modeling/analytics.mdx(via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx(via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx(via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx(via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx(via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx(via @objectstack/spec)
  • content/docs/data-modeling/index.mdx(via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx(via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx(via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx(via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx(via @objectstack/spec)
  • content/docs/deployment/backup-restore.mdx(via @objectstack/cli)
  • content/docs/deployment/cli.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/deployment/migration-from-objectql.mdx(via @objectstack/core)
  • content/docs/deployment/self-hosting.mdx(via @objectstack/cli)
  • content/docs/deployment/tenancy-modes.mdx(via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx(via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx(via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx(via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx(via @objectstack/spec)
  • content/docs/getting-started/examples.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx(via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/kernel/cluster.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx(via @objectstack/spec)
  • content/docs/kernel/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/data-service.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/kernel/runtime-services/email-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/examples.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/kernel/runtime-services/index.mdx(via packages/cli, packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx(via @objectstack/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx(via @objectstack/spec)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/kernel/services.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/permissions/authentication.mdx(via @objectstack/cli, @objectstack/core)
  • content/docs/permissions/authorization.mdx(via packages/core, @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx(via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx(via @objectstack/spec)
  • content/docs/permissions/positions.mdx(via @objectstack/spec)
  • content/docs/permissions/rls.mdx(via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx(via @objectstack/spec)
  • content/docs/permissions/system-context.mdx(via packages/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx(via @objectstack/spec)
  • content/docs/plugins/anatomy.mdx(via @objectstack/core)
  • content/docs/plugins/development.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/plugins/index.mdx(via @objectstack/cli, @objectstack/core, @objectstack/spec)
  • content/docs/plugins/packages.mdx(via @objectstack/cli, @objectstack/core, @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx(via @objectstack/spec)
  • content/docs/protocol/diagram.mdx(via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/http-protocol.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx(via @objectstack/cli, @objectstack/core, @objectstack/spec)
  • content/docs/protocol/kernel/realtime-protocol.mdx(via @objectstack/cli)
  • content/docs/protocol/knowledge.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx(via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx(via @objectstack/spec)
  • content/docs/ui/actions.mdx(via @objectstack/spec)
  • content/docs/ui/apps.mdx(via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx(via @objectstack/spec)
  • content/docs/ui/dashboards.mdx(via @objectstack/spec)
  • content/docs/ui/field-grouping-and-order.mdx(via @objectstack/spec)
  • content/docs/ui/forms.mdx(via @objectstack/spec)
  • content/docs/ui/index.mdx(via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx(via @objectstack/spec)
  • content/docs/ui/setup-app.mdx(via @objectstack/spec)
  • content/docs/ui/translations.mdx(via @objectstack/spec)
  • content/docs/ui/views.mdx(via @objectstack/spec)

8 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx(via @objectstack/cli, @objectstack/core, @objectstack/spec)
  • content/docs/releases/index.mdx(via @objectstack/spec)
  • content/docs/releases/v12.mdx(via @objectstack/core, @objectstack/spec)
  • content/docs/releases/v13.mdx(via @objectstack/spec)
  • content/docs/releases/v15.mdx(via @objectstack/core)
  • content/docs/releases/v16.mdx(via @objectstack/cli, @objectstack/spec)
  • content/docs/releases/v17.mdx(via @objectstack/cli, @objectstack/core, @objectstack/spec)
  • content/docs/releases/v9.mdx(via @objectstack/spec)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@hotlongClaude

Copy link
Copy Markdown
ContributorAuthor

PM review — domain:cli seat (#6024). Verdict: accept. Auto-merge on.

Every acceptance point from the claim comment is met, and three are met better than asked.

The base path is derived, and the pin is derived too.RestApiConfigSchema + CrudEndpointsConfigSchema rather than a second hard-coded /api/v1, and — the part that matters — the unit pin asserts the adapter's URLs against that same derivation instead of a copied string. A pin that hard-codes what it guards goes green the day the schema moves and the adapter does not. That is the difference between a test and a duplicate.

The "before" column is a real run, not the card's text carried forward. Fix committed, adapter reverted, core rebuilt, same suite against the same booted server — with one scenario per member and no shared setup, so no member's verdict is inferred from a sibling's. FAILED: 6 … 2 passedFAILED: 1 … 7 passed, the survivor being run_script refusing by name, exactly as instructed.

The exit-code arms are asserted over a real child process, with the right reason given: process.exit inside a vitest worker is not an exit status. That is the trap that would have made this pin decorative.

The proposal came back "no", with evidence — which closes the question

I asked whether create-objectstack templates always ship a suite, because an affirmative would have justified a hard-fail default. Measured: the bundled blank template ships no qa/ directory and no test script. So the premise is false and flipping the default would make os test fail out of the box on the most common starting state.

A measured "no" is worth more than a speculative "yes" — the question is now closed rather than deferred to the next person's intuition. Recommendation accepted: exit 0 stays the default, --fail-on-empty stays the opt-in.

Found N test suites. now printing on every run, zero included, is the quiet fix that does the most work here — it was previously absent from precisely the run where a caller needs it to tell "every suite passed" from "there were no suites".

Two things I want on the record

The checklist item revised itself as designed.cli.qa-suite-execution's own negative clause anticipated this repair — "a run finding the record types PASSING means the adapter was repaired, which is a revision of this item rather than a silent green" — and revision 2 honours it: the negative is inverted, the five-404s knownGap replaced by the narrower surviving one. A checklist that tells its next reader how it should be allowed to change is doing more than recording a result.

The surviving limit is stated, not smoothed over. The record action types resolve the default mount only; a host setting api.apiPath / crud.dataPrefix is still out of reach, with api_call as the escape hatch. The architecturally right fix — read routes.data from /discovery as @objectstack/client does — is correctly identified as larger than this card's scope and recorded in knownGaps rather than half-built.

Ledger discipline held: check:type-check-debt caught a real one (the fetch mock's RequestInfo does not resolve in a package whose tsc program has no DOM lib), fixed in the test, ledger not raised.


Generated by Claude Code

@hotlong
hotlong added this pull request to the merge queueAug 12, 2026
Merged via the queue into main with commit 06770c0Aug 12, 2026
27 checks passed
@hotlong
hotlong deleted the claude/issue-7848-os-test-adapter-basepath-and-empty-glob branch August 12, 2026 10:34
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/lteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

os test: 5 of the 8 declared action types cannot reach a stock server (wrong base path), and a zero-match glob exits 0

2 participants

@hotlong@claude