Skip to content

feat(dogfood): in-process runtime regression gate - #2020

Merged
os-zhuang merged 1 commit into
mainfrom
feat/dogfood-regression-gate
Jun 18, 2026
Merged

feat(dogfood): in-process runtime regression gate#2020
os-zhuang merged 1 commit into
mainfrom
feat/dogfood-regression-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Why

Static gates — build, unit tests, spec-liveness, CodeQL — verify each layer in isolation, usually against mocks. They cannot catch a break that only appears when the real engine + strategies + services + HTTP context run together.

#2018 ("org timezone drives analytics date bucketing") is the canonical case: it was green on every static gate — 900+ unit tests included — because each of its three broken seams (analytics strategy routing, in-memory count, REST execution-context) was individually correct and individually mocked. The break was only visible by booting the app and comparing a date bucket under UTC vs a non-UTC org timezone. Unit tests that mock the protocol/server (e.g. rest.test.ts) structurally cannot catch it.

What

@objectstack/dogfood (private, never published): a harness that boots real example apps in-process against in-memory SQLite — wired with the same service plugins objectstack dev loads — and exercises them through the real HTTP surface via Hono request-injection (no ports, no sockets, CI-stable). Tests act as a browser client: sign in, hit /api/v1/..., assert on real responses.

  • src/harness.tsbootDogfoodStack(config){ kernel, api, raw, signIn, apiAs, stop }. Registers ObjectQL + SQLite + HonoServer + App + Settings + Analytics + Auth + Security + REST + Dispatcher, runs kernel.bootstrap() (seed + dev-admin + route registration), and injects requests through the live Hono app.
  • test/analytics-timezone.dogfood.test.ts — golden regression for ADR-0053 Phase 2 · Slice 5: timezone-aware analytics date bucketing #1982/fix(analytics): make organization timezone drive date-dimension bucketing (#1982) #2018. Inserts leads at a tz-boundary instant (2024-03-01T03:00:00Z), sets the org timezone through the real settings route, and asserts the date bucket shifts (UTC → 2024-03-01, America/Los_Angeles → 2024-02-29) with the correct count.
  • CI — new Dogfood Regression Gate job (gated on the core paths filter); also runs under turbo run test.

Proof it's a real gate

Reverting just one of the three #2018 seams (NativeSQLStrategy routing) and rebuilding turns the LA assertion red (expected undefined to be 3 — the bucket didn't shift). Restoring it → green. A gate that stays green on the bug is not a gate; this one fails on it.

Boot-to-assert ≈ 2s. Private package — no changeset.

Notes

🤖 Generated with Claude Code

Static gates (build, unit tests, spec-liveness, CodeQL) verify each layer in
isolation, usually against mocks, and cannot catch a break that only appears
when the real engine + strategies + services + HTTP context run together. The
#2018 tz-bucketing regression is the canonical case: green on every static gate
(900+ unit tests included) because each of its three broken seams was
individually correct and individually mocked.
This adds `@objectstack/dogfood` (private), a harness that boots real example
apps in-process against in-memory SQLite — wired with the same service plugins
`objectstack dev` loads — and exercises them through the real HTTP surface via
Hono request-injection (no ports, no sockets, CI-stable). Tests act as a browser
client: sign in, hit /api/v1/..., assert on real responses.
- src/harness.ts — bootDogfoodStack(config) → { kernel, api, raw, signIn, apiAs, stop }
- test/analytics-timezone.dogfood.test.ts — golden regression for #1982/#2018:
org timezone (set via the real settings route) must shift a boundary-crossing
instant's date bucket (UTC 2024-03-01 → America/Los_Angeles 2024-02-29) with
the correct count. Verified to FAIL when any #2018 seam is reverted.
- CI: new "Dogfood Regression Gate" job (also runs under turbo run test).
Boot-to-assert ~2s. Private package — no changeset.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercelBot commented Jun 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
specReadyReadyPreview, CommentJun 18, 2026 5:31am

Request Review

@github-actionsgithub-actionsBot added size/m documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tests tooling labels Jun 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

No hand-written docs reference the 1 changed package(s). ✅

@os-zhuang
os-zhuang merged commit ebc7ecb into mainJun 18, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the feat/dogfood-regression-gate branch June 18, 2026 05:36
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filedocumentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@os-zhuang