Uh oh!
There was an error while loading. Please reload this page.
Canonicalise driver-materialised timestamps at the metadata adapter boundaries - #14040
Conversation
…pter boundaries
`MetadataItem.authoredAt` is declared `z.string()` ('ISO-8601 timestamp') and
`MetadataStats.mtime` is declared `z.string().datetime()`. Three producers
adapted a driver row into those declared types without converting the value:
- `SysMetadataRepository#getByHash` — `recorded_at`, a declared
`Field.datetime` on `sys_metadata_history`
- `SysMetadataRepository#rowToItem` (via `#get`) — the builtin audit columns
- `DatabaseLoader#stat` — the same audit columns, through `rowToRecord`
`SqlDriver#formatOutput` repairs the audit columns and folds declared datetime
columns only inside its `if (this.isSqlite)` arm, and
`withPostgresCalendarDayAsText` leaves `timestamptz` / `timestamp` deliberately
untouched — so on Postgres and MySQL a JS `Date` landed in a field every
consumer reads as a `string`. Nothing reported it: `row` is `any`, and
`MetadataItemSchema` is parsed nowhere on a production path.
Canonicalised at the producer, matching the spelling `auditMetaItem` already
applies to `sys_metadata_audit.occurred_at`. Already-canonical SQLite text
passes through byte-identically. Pinned by driving a hand-made `Date` through
each adapter, asserted through `MetadataItemSchema` itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L… double, and record its pins Two gates named this file, both correctly: - `check:objectql-double-limit` — the fake `find` ignored `opts.limit`, so it would answer more rows than the real engine. Now applies the caller's bound AFTER the filter and BY PRESENCE, which is the conforming shape (the gate's graded population moves 98 -> 99 applying the bound; no baseline grows). - `check:engine-double-contract` — the new double's three write/read verbs are pinned to ObjectQL's dispatch predicates but the RETAINED ledger did not know the file yet, so nothing protected it. Regenerated with `--write`: 3 rows added or grown, 0 lost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F3jdziLbAPGeceVNmSox5L
📓 Docs Drift Check3 anchor(s) derived from 2 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 12 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 28683db0f84523bed42e36287b3608ceb55cef41 && git checkout 28683db0f84523bed42e36287b3608ceb55cef41
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc 9caf55e082f59e9bfdb686dafd93e783f6bd6bc5 && git checkout -B drift-repro 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc && git merge --no-ff 9caf55e082f59e9bfdb686dafd93e783f6bd6bc5
node scripts/docs-audit/affected-docs.mjs --json 5e2c04da7db38c4db0b138fad8b9be5b4ef308fc |
Uh oh!
There was an error while loading. Please reload this page.
Fixes#13997
MetadataItem.authoredAtis declaredz.string().describe('ISO-8601 timestamp')andMetadataItemis az.infer, so the field isstringto every consumer.MetadataStats.mtimeis declaredz.string().datetime(). Three producers adapted a driver row into those declared types without converting the value, and on Postgres and MySQL a JSDatelanded in each of them.The mechanism
created_at/updated_atare builtin audit columns andrecorded_atis a declaredField.datetime.SqlDriver#formatOutputrepairs the audit columns (repairNaiveUtcAuditTimestamp) and folds declared datetime columns (normalizeSqliteDatetimeOutput) only inside itsif (this.isSqlite)arm, andwithPostgresCalendarDayAsTextleaves the instant types alone on purpose: "timestamptz/timestampare deliberately untouched: those are instants, aDateis the right materialisation for them, andField.datetimedepends on it." Pinned live inpackages/drivers/driver-sql/src/sql-driver-13567-audit-stamp-materialisation.test.ts.Two independent reasons nothing reported it: the row is
any, so tsc saw astringassignment that never happened; and the runtime validator that would have caught it never runs on these paths (see the observation at the end).What changed
Canonicalised at the producer — the adapter boundary that asserts the declared type — matching the spelling
auditMetaItemalready applies tosys_metadata_audit.occurred_atinprotocol.ts. Not a third spelling, and not a tolerant fallback: it converts the one per-dialect materialisation the driver genuinely produces into the single declared spelling. Values that were already canonical (SQLite) pass through byte-identically.sys-metadata-repository.tsgetByHash()MetadataItem.authoredAt(row as any).recorded_at ?? …sys-metadata-repository.tsrowToItem()(viaget())MetadataItem.authoredAtrow.updated_at ?? row.created_at ?? …database-loader.tsstat()MetadataStats.mtimerecord.updatedAt ?? record.createdAt ?? …Route B — normalising at the driver's read door — is deliberately not taken: it reverses a stated driver decision and belongs to the whole census, not to this card.
The issue and its triage both list
sys-metadata-repository.ts:417(getByHash) among the five sibling producers that already spell it canonically, on the stated ground thatrecorded_atis a declaredField.datetimerather than a builtin audit column. Measured, that ground does not hold: the declared-Field.datetimecoercion is SQLite-gated too. Inpackages/drivers/driver-sql/src/sql-driver.tsthedatetimeFieldsnormalisation loop sits at brace depth 2 inside theisSqlitearm opened at line 15879 and closed at 15968 — andwithPostgresCalendarDayAsTextsays in as many words thatField.datetimedepends on theDatematerialisation.So
getByHashwas a third straight-through site, not a canonical sibling. It is in one of the two files the card names and carries the identical defect on the identical declared field, so it is repaired here rather than deferred. Everything else measured in this sweep is filed separately (below).The card's central contrast survives and is stronger than stated — it is 9 canonical producers against 3 straight-through ones, and the straight-through ones are still exactly the sites that pass a driver row through unconverted:
sys-metadata-repository.ts:606,in-memory-repository.ts:120,metadata-fs/src/repository.ts:271,:300,:374,:421(all from a JSONL log field written asthis.now().toISOString()),memory-loader.ts:72,remote-loader.ts:108,filesystem-loader.ts:227sys-metadata-repository.ts:417,:1752,database-loader.ts:917The pin
packages/metadata-protocol/src/sys-metadata-repository-13997-authored-at-canonicalisation.test.ts, plus two cases appended todatabase-loader.test.ts.The existing schema test proves nothing about this seam because it feeds a hand-made string — the assertion and the input share an identity. Every case here drives a hand-made
Dateinstead, the one shape the live dialects produce and no existing fixture ever did, and each carries a non-vacuity guard asserting the input really is aDatebefore the output is read. There is no driver dependency:metadata-protocolhas none and must not grow one, so theDateis hand-made for the same reason the #13567 pin states for the OCC seam next door. TheauthoredAtcases assert throughMetadataItemSchema.safeParseitself rather than a hand-rolled regex.Ablation, against the committed tree, restored under a trap on absolute paths: reverting the three call sites turns the pins red — 2 of 3 in the new file, 1 in
database-loader.test.ts— withAssertionError: expected 'object' to be 'string', which is the defect verbatim. The idempotence cases stayed green under the ablation, which is what shows the pins discriminate rather than being globally sensitive. Restore verified by an emptygit diff HEADand blob hashes identical toHEAD.Gates
Run at
9caf55e082, exit codes captured before any pipe.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstacknamed 30 families, re-derived after the final commit (the ledger edit pulled in 7 more, all run and green).Two went red and both were mine, each repaired rather than baselined:
check:objectql-double-limit— the new fakefindignoredopts.limit. Now applies the caller's bound after the filter and by presence; the gate's graded population moves from 98 to 99 applying the bound, and no baseline grew.check:engine-double-contract— the new double's three verbs are pinned to ObjectQL's dispatch predicates, but the RETAINED ledger did not know the file, so nothing protected it. Regenerated with--write: 3 rows added or grown, 0 lost. They are pinned rows, not baseline entries.check:type-check-debt --re-measurepasses against the built workspace closure: 29 ledger entries re-measured, none above its recorded number, "surplus: none — every entry sits exactly at its measurement", so the new test file adds zero type errors. Both affected packages carry type-check DEBT ledger entries and declare notypecheckscript, so a per-package typecheck is not available here and this ratchet is the real type evidence.Tests:
@objectstack/metadata627/627,@objectstack/metadata-protocol2063 passed with 10 pre-existing skips.Not measured, and reported as such rather than as passes:
check-test-completeness(exit 3 — it parses a CI test-run summary that does not exist locally).check:dual-build-cjs-loadsinitially exit 3 and green once the closure was built.MetadataItemSchemais a declared runtime validator that exists, is trusted, and has zero production coverage: its only.parsecall sites in the repo are its own unit test (packages/metadata-core/test/types.test.ts:70,74,78), which feeds a hand-made sample, so it evaluates only against inputs that were never near a driver — which is exactly why it did not catch this. Whether it should be parsed somewhere on a real path, and whether other declared schemas are in the same state, is a larger question than this card and is not answered here.That zero is measured with a firing positive control, in the same file: the sibling
MetadataEventSchema, declared beside it inpackages/metadata-core/src/types.tsand exported from the same barrel, is parsed on production paths (packages/metadata/src/metadata-manager.ts:693,packages/client/src/realtime-api.ts:107), and the same expression family finds 289 hits acrosspackages/. So the zero is a fact about this schema, not an artefact of the expression or the search scope.Filed separately, not addressed here
#14037 — five further adapter-boundary sites in these same two files cast a driver
Dateinto a declared ISO-string timestamp (MetadataEvent.ts,MetadataHistoryRecord.recordedAt,MetadataRecord.createdAt/updatedAt). Three of them are declaredz.string().datetime(), stricter than the field this card dealt with. Scope for that card, not this one.#13973 (the census) and #13382 (the OCC seam) remain open and are not touched by this change.
Generated by Claude Code