fix(driver-sql): Field.time 规范化存储 — HH:MM:SS[.fff] 墙钟文本，三方言统一 (#3994)

.changeset/field-time-canonical-storage.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/driver-sql": minor
+"@objectstack/cli": patch
+"@objectstack/spec": patch
+---
+
+fix(driver-sql): `Field.time` gets a canonical storage form — `HH:MM:SS[.fff]` wall-clock text on every dialect (#3994)
+
+`Field.time` repeated the pre-#3912 `Field.datetime` pattern: writes were never
+normalised and only reads were repaired, so one SQLite column accumulated bare
+time-of-day TEXT, full-timestamp TEXT and INTEGER epoch ms side by side.
+`find()` looked right; everything that compared the STORED form was wrong —
+measured: a business-hours window filter silently dropped 4 of 7 rows, ORDER BY
+sorted 14:30 before 08:00, a full-ISO write failed the statement outright on
+both Postgres and MySQL, a bound `Date` stored a process-timezone wall clock on
+pg, MySQL's bare `TIME` rounded `…00.500` up to `…01`, and a `NOW()` default
+resolved against three different clocks on the three dialects.
+
+The #3912→#3942→#3954 construction, transplanted (ADR-0053 D-C1..D-C3):
+
+- One `canonicalTimeOfDay` — `HH:MM:SS`, `.fff` only when non-zero; `Date`/
+ epoch/full-timestamp fold to the UTC time-of-day — applied on write
+ (`formatInput`), to filter comparands (`coerceFilterValue`, and thereby the
+ `temporalFilterValue` contract hook) and on read (`toTimeOnly`).
+- SQLite: legacy columns converge at schema sync (`backfillCanonicalTimes`,
+ same `IS NOT`-guarded UPDATE, same log-and-swallow policy); until then the
+ filter paths wrap the column in the repair expression — correct, just
+ unindexed. `os migrate plan` lists the work as `normalize_time_storage` with
+ a row count.
+- MySQL: new time columns are `TIME(3)`; legacy `TIME(0)` columns widen at
+ schema sync (`migrateMysqlTimeColumns`, plan kind `widen_time_columns`),
+ since zero-precision TIME *rounds* fractional writes.
+- `NOW()` defaults read the UTC clock on every dialect (Postgres previously
+ used the server zone, MySQL the inserting session's zone — and MySQL 8.0
+ rejects a plain `CURRENT_TIMESTAMP` default on TIME entirely).
+- `distinct()`/`aggregate()` present time columns exactly as `find()` does.
+
+`HH:MM:SS` writes round-trip byte-identically (the field-zoo `f_time`
+contract); a minutes-only `HH:MM` now completes to `HH:MM:00`, and uninterpretable
+values still pass through untouched.

docs/adr/0053-date-and-datetime-semantics.md

@@ -1,6 +1,6 @@
 # ADR-0053: `date` is a timezone-naive calendar day; `datetime` is an instant rendered in a reference timezone
 
-**Status**: Accepted (2026-06-16) — Phase 1 + addendum D-A1 implemented (`sql-driver.ts` `toDateOnly` write/read/filter normalization; analytics `coerceTemporalFilterValue`), Phase 2 landing incrementally; D-A2 resolved 2026-07-30: `temporalFilterValue` + `temporalFilterColumnSql` are optional `IDataDriver` contract members with identity semantics, and analytics types its driver seam from the contract. **Partly superseded (2026-07-29, addendum D-B1..D-B4):** Phase 1's "`Field.datetime` stays stored as UTC epoch ms" is replaced by one canonical UTC instant per dialect — `YYYY-MM-DDTHH:MM:SS.sssZ` text on SQLite, `timestamptz` on Postgres, `DATETIME(3)` on MySQL — applied on write and to filter comparands alike (#3912, #3942).
+**Status**: Accepted (2026-06-16) — Phase 1 + addendum D-A1 implemented (`sql-driver.ts` `toDateOnly` write/read/filter normalization; analytics `coerceTemporalFilterValue`), Phase 2 landing incrementally; D-A2 resolved 2026-07-30: `temporalFilterValue` + `temporalFilterColumnSql` are optional `IDataDriver` contract members with identity semantics, and analytics types its driver seam from the contract. **Partly superseded (2026-07-29, addendum D-B1..D-B4):** Phase 1's "`Field.datetime` stays stored as UTC epoch ms" is replaced by one canonical UTC instant per dialect — `YYYY-MM-DDTHH:MM:SS.sssZ` text on SQLite, `timestamptz` on Postgres, `DATETIME(3)` on MySQL — applied on write and to filter comparands alike (#3912, #3942). **Extended (2026-07-30, addendum D-C1..D-C3):** `Field.time` takes the same construction — canonical `HH:MM:SS[.fff]` wall-clock text, one function on write/filter/read, `TIME(3)` on MySQL, UTC `NOW()` defaults on every dialect (#3994).
 **Deciders**: ObjectStack Protocol Architects
 **Builds on**: [ADR-0032](./0032-unified-expression-layer.md) (unified expression layer — CEL dialect, `today()`/`daysFromNow()`), [ADR-0014](./0014-record-form-field-type.md) (field types)
 **Consumers**: `@objectstack/spec` (`Field.date`/`Field.datetime`), `@objectstack/driver-sql` (`coerceFilterValue`, `formatInput`/`formatOutput`, `dateFields`/`datetimeFields`), `@objectstack/formula` (`stdlib` time functions, `cel-engine` hydration), `@objectstack/objectql` (`applyFormulaPlan`), schedule/cron executors, report/analytics date bucketing, `sys-user-preference.timezone`.
@@ -574,3 +574,78 @@ The same caveat as Postgres applies and is worth stating plainly: the migration
 Regression cover is `sql-driver-datetime-mysql-storage.test.ts`, opt-in via
 `OS_TEST_MYSQL_URL` (CI provisions no server), asserting a non-UTC server so it
 cannot pass vacuously. 10 of its 13 cases fail without this change.
+
+---
+
+## Addendum (2026-07-30) — `Field.time` gets the same storage convention (D-C1..D-C3, #3994)
+
+`Field.time` was the last temporal field type with **no storage convention at
+all** — the exact meta-problem #3912 exposed for `Field.datetime`. `formatInput`
+never touched it, `coerceFilterValue` returned its comparands unchanged, and the
+read paths papered over the drift so well (`toTimeOnly`) that `find()` always
+looked right. Measured on real servers (SQLite; PG 16 at `Asia/Shanghai`;
+MariaDB 10.11 / MySQL 8.0 at `+08:00`; Node at `America/New_York`), the same
+failure family followed: a business-hours window filter silently dropped 4 of 7
+rows on SQLite (INTEGER epoch rows fail `>= '09:00'` because `INTEGER < TEXT`;
+full-timestamp text fails `<= '18:00'` lexicographically), ORDER BY put 14:30
+before 08:00, a full-ISO write failed the statement outright on PG **and**
+MySQL, a bound `Date` stored a process-timezone wall clock on pg, MySQL's bare
+`TIME` rounded `…00.500` up to `…01`, and a `NOW()` default resolved against
+three different clocks on the three dialects.
+
+### D-C1 — The canonical form is `HH:MM:SS`, `.fff` only when non-zero
+
+A `Field.time` is a **timezone-naive wall-clock time-of-day** (#2004), so the
+canonical text carries no zone: `HH:MM:SS`, extended to `HH:MM:SS.fff` exactly
+when the milliseconds are non-zero. One function (`canonicalTimeOfDay`) produces
+it and is applied on write (`formatInput`), to filter comparands
+(`coerceFilterValue`/`temporalFilterValue`) and on read (`toTimeOnly`), the
+D-B1 construction transplanted.
+
+Why variable-width rather than datetime's fixed `.sss`: `.` sorts below every
+digit, so lexicographic order is still chronological order across mixed widths
+(`'14:30:00.100' < '14:30:01'`), and the zero-millisecond spelling `HH:MM:SS`
+is what every dialect's native TIME emits — the field-zoo `f_time` round-trip
+(#2022) keeps holding byte-for-byte. Determinism is what matters for equality
+and `distinct()`, and each wall clock has exactly one spelling. A minutes-only
+`'14:30'` completes to `'14:30:00'`.
+
+A `Date` / epoch-ms / full-timestamp value folds to its **UTC** time-of-day —
+matching the platform's instant semantics, the SQLite read repair's historical
+behaviour, and `nowColumnDefault`; never the process or server timezone.
+Uninterpretable values (including out-of-range wall clocks like `'25:00'`) pass
+through untouched.
+
+### D-C2 — Physical form per dialect
+
+- **SQLite**: canonical TEXT (no native type). Legacy columns converge at
+ schema sync via `backfillCanonicalTimes` — one `UPDATE` per column whose SET
+ expression IS the read-repair SQL (`sqliteCanonicalTimeSql`), `IS NOT` guard,
+ log-and-swallow failure policy, `os migrate plan` row-counted entry
+ (`normalize_time_storage`) — D-B3 verbatim. Until it runs, filters wrap the
+ column in the repair expression: correct, just unindexed.
+- **Postgres**: native `time` (µs precision). The canonical literal binds
+ unambiguously; the pre-fix failures were the *input* shapes, not the column.
+- **MySQL**: `TIME(3)` — the `DATETIME(3)` precedent applied: bare `TIME` is
+ zero-precision and **rounds** fractional literals, changing the stored wall
+ clock. Legacy `TIME(0)` columns widen at schema sync
+ (`migrateMysqlTimeColumns`, plan kind `widen_time_columns`), restating a
+ `NOW()` expression default where declared.
+
+### D-C3 — `NOW()` defaults are the UTC wall clock on every dialect
+
+`knex.fn.now()` compiles to `CURRENT_TIMESTAMP`, which a TIME column resolves
+in the **server's** zone on Postgres and the **inserting session's** zone on
+MySQL (so the same column's default depended on who inserted). MySQL 8.0
+additionally rejects a plain `CURRENT_TIMESTAMP` default on TIME. The defaults
+are now expression-spelled per dialect — `strftime('%H:%M:%f','now')` with a
+canonical `.000` trim on SQLite, `timezone('utc', now())::time(3)` on Postgres,
+`(cast(utc_timestamp(3) as time(3)))` on MySQL (8.0.13+/MariaDB 10.2+ for the
+expression-default syntax) — all reading the UTC clock.
+
+The D-B3 caveat applies unchanged: the backfill converges *shapes*; a wall
+clock the old path never recorded correctly (a pg `Date` bind serialised in the
+host's zone) is not recoverable. Regression cover:
+`sql-driver-time-canonical-storage.test.ts`, `sql-driver-time-of-day.test.ts`
+(SQLite), `sql-driver-time-live-dialects.test.ts` (live PG + MySQL, in the CI
+temporal-conformance job's non-UTC matrix).

packages/cli/src/utils/schema-migrate.pending-render.test.ts

@@ -41,6 +41,8 @@ const ADDITIVE: PendingSchemaWork[] = [
 const IN_PLACE: PendingSchemaWork[] = [
  { table: 'evt', kind: 'normalize_datetime_storage', columns: ['at'], rows: 1234567 },
  { table: 'legacy', kind: 'widen_datetime_columns', columns: ['at', 'created_at'], rows: 42 },
+ { table: 'shift', kind: 'normalize_time_storage', columns: ['starts_at'], rows: 7 },
+ { table: 'shift_my', kind: 'widen_time_columns', columns: ['starts_at'], rows: 9 },
 ];
 
 describe('renderPendingSchemaWork (#3954)', () => {
@@ -74,6 +76,11 @@ describe('renderPendingSchemaWork (#3954)', () => {
  expect(out()).toContain('widen_datetime_columns: at, created_at');
  // A MySQL widen is ALTER … MODIFY — a rebuild, said outright.
  expect(out()).toContain('42 row table rebuild');
+ // The Field.time twins (#3994) take the same rendering rules.
+ expect(out()).toContain('normalize_time_storage: starts_at');
+ expect(out()).toContain('7 row update(s)');
+ expect(out()).toContain('widen_time_columns: starts_at');
+ expect(out()).toContain('9 row table rebuild');
  });
 
  it('shows both sections when both kinds are pending', () => {
@@ -102,7 +109,7 @@ describe('summarizePendingSchemaWork (#3954)', () => {
  const summary = summarizePendingSchemaWork([...ADDITIVE, ...IN_PLACE]);
  expect(summary).toContain('1 table(s) to create');
  expect(summary).toContain('1 column(s) to add');
- expect(summary).toContain('3 datetime column(s) to converge in place');
- expect(summary).toContain('~1,234,609 rows');
+ expect(summary).toContain('5 temporal column(s) to converge in place');
+ expect(summary).toContain('~1,234,625 rows');
  });
 });

packages/cli/src/utils/schema-migrate.ts

@@ -285,17 +285,14 @@ export function renderPendingSchemaWork(pending: PendingSchemaWork[]): void {
  if (inPlace.length > 0) {
  console.log(` ${chalk.bold('In place (existing rows converged when you apply)')}`);
  for (const p of inPlace) {
- const label = p.kind === 'normalize_datetime_storage'
- ? 'normalize_datetime_storage'
- : 'widen_datetime_columns';
  // A MySQL widen is `ALTER … MODIFY`, i.e. a full table rebuild holding a
  // metadata lock — worth saying outright, not just implying via the count.
- const cost = p.kind === 'widen_datetime_columns'
+ const cost = p.kind === 'widen_datetime_columns' || p.kind === 'widen_time_columns'
  ? `${formatRows(p.rows)} row table rebuild`
  : `${formatRows(p.rows)} row update(s)`;
  console.log(
  ` ${chalk.yellow('~')} ${chalk.yellow(p.table)} ` +
- `${chalk.dim(`[${label}: ${p.columns.join(', ')} — ${cost}]`)}`,
+ `${chalk.dim(`[${p.kind}: ${p.columns.join(', ')} — ${cost}]`)}`,
  );
  }
  console.log('');
@@ -320,7 +317,7 @@ export function summarizePendingSchemaWork(pending: PendingSchemaWork[]): string
  if (inPlace.length > 0) {
  const cols = inPlace.reduce((n, p) => n + p.columns.length, 0);
  const rows = inPlace.reduce((n, p) => n + (p.rows ?? 0), 0);
- parts.push(`${cols} datetime column(s) to converge in place (~${formatRows(rows)} rows)`);
+ parts.push(`${cols} temporal column(s) to converge in place (~${formatRows(rows)} rows)`);
  }
  return parts.join(', ');
 }

packages/plugins/driver-sql/src/legacy-datetime-storage.testkit.ts

@@ -39,6 +39,24 @@ export class LegacyStorageDriver extends SqlDriver {
  this.canonicalDatetimeFields[table]?.delete(field);
  }
 
+ /**
+ * The `Field.time` twin (#3994): insert raw pre-canonical time values and
+ * clear the column's canonical marker.
+ */
+ async seedLegacyTimeRows(
+ table: string,
+ field: string,
+ rows: Array<Record<string, unknown>>,
+ ): Promise<void> {
+ await this.knex(table).insert(rows);
+ this.forgetCanonicalTime(table, field);
+ }
+
+ /** Drop the "already backfilled" marker for one `Field.time` column. */
+ forgetCanonicalTime(table: string, field: string): void {
+ this.canonicalTimeFields[table]?.delete(field);
+ }
+
  /** Raw stored form of a column, for asserting on the fixture's premise. */
  async storedForms(table: string, field: string): Promise<Array<{ id: string; type: string; value: unknown }>> {
  const res: any = await this.knex.raw(

packages/plugins/driver-sql/src/schema-drift.ts

@@ -128,23 +128,26 @@ export interface PendingSchemaWork {
 /**
  * What kind of physical work a {@link PendingSchemaWork} entry represents.
  *
- * The first two are purely additive and never touch existing rows. The datetime
- * pair is NOT: `normalize_datetime_storage` rewrites rows in place (the SQLite
- * canonical-UTC backfill) and `widen_datetime_columns` rebuilds a column (the
- * MySQL `TIMESTAMP` → `DATETIME(3)` widening) — both from #3912/#3942. They are
- * rendered under their own heading for that reason: the additive section tells
- * the operator the work is never data-losing, and that claim must not silently
- * come to cover a row rewrite.
+ * The first two are purely additive and never touch existing rows. The rest are
+ * NOT: `normalize_datetime_storage` / `normalize_time_storage` rewrite rows in
+ * place (the SQLite canonical-text backfills, #3912/#3994) and
+ * `widen_datetime_columns` / `widen_time_columns` rebuild a column (the MySQL
+ * `TIMESTAMP` → `DATETIME(3)` and `TIME` → `TIME(3)` widenings, #3942/#3994).
+ * They are rendered under their own heading for that reason: the additive
+ * section tells the operator the work is never data-losing, and that claim must
+ * not silently come to cover a row rewrite.
  */
 export type PendingSchemaWorkKind =
  | 'create_table'
  | 'add_columns'
  | 'normalize_datetime_storage'
- | 'widen_datetime_columns';
+ | 'normalize_time_storage'
+ | 'widen_datetime_columns'
+ | 'widen_time_columns';
 
 /** True for the kinds that rewrite or rebuild existing data rather than adding to it. */
 export function isInPlaceSchemaWork(kind: PendingSchemaWorkKind): boolean {
- return kind === 'normalize_datetime_storage' || kind === 'widen_datetime_columns';
+ return kind !== 'create_table' && kind !== 'add_columns';
 }
 
 /** Ops that act on an index rather than a column — reconciled without a table rebuild. */