feat(cli): os migrate duplicates reports the rows blocking the kernel:ready index tightenings

.changeset/migrate-duplicates-kernel-ready-preflight.md

@@ -0,0 +1,62 @@
+---
+"@objectstack/metadata-protocol": minor
+"@objectstack/cli": minor
+---
+
+`os migrate duplicates` now reports the rows blocking the three `kernel:ready`
+NULL-safe index tightenings, and the three migrations' conflict messages point
+there instead of at `os migrate plan` (#8725).
+
+**The gap.** Three migrations replace a declared UNIQUE index with the NULL-safe
+— and sometimes active-rows-only — form it was always meant to have, at
+`kernel:ready` on a serving boot:
+
+| table | index(es) | migration |
+| --- | --- | --- |
+| `sys_metadata` | overlay `active` + `draft` | `ensureMetadataOverlayIndexes` |
+| `sys_view_definition` | `idx_sys_view_def_active` | `ensureViewDefinitionActiveIndex` |
+| `sys_setting` | the declared row identity | `ensureSysSettingIdentityIndex` |
+
+Each is a tightening, so rows an installation already holds can block it. The
+migration then refuses — previous index kept, no row touched, boot continues —
+and reports at `error` on the boot channel. That channel was the only one:
+these indexes are invisible to `os migrate plan` **by construction**, twice
+over. After the tightening, `isRuntimeManagedIndex` excludes the index (without
+that exclusion a boot would propose rebuilding away the guarantee it had just
+created); before it, each migration deliberately reuses the *declared* index's
+name, so the reconciler's name-matched slot reads as filled whichever physical
+form is really there. Measured with a matched control — one database carrying
+the same duplicate damage under a declared index and under
+`sys_view_definition`'s runtime one — `plan` named the declared one in full and
+said nothing whatsoever about the runtime one.
+
+**What is new.** The report gains a `runtimeIndexPreflight` section, one entry
+per index, each `blocked` (with every colliding key group and its row count),
+`clear`, `table-absent` (`sys_setting` arrives with the optional settings
+service) or `unreadable` (with the driver's own message), plus
+`summary.runtimeIndexesBlocked` and `summary.runtimeIndexBlockingRows`.
+`reportVersion` moves `1` → `2`. Every `1` field keeps its name, shape and
+meaning; the bump says there is more in the document, for consumers that
+validate it strictly.
+
+The probes are the migrations' own duplicate-listing statements —
+`@objectstack/metadata-protocol` exports `collectRuntimeIndexPreflight` and
+`runtimeIndexProbes`, which read those builders rather than restating the keys,
+so the pre-flight and the boot report cannot describe different duplicates. On
+MySQL the `sys_setting` probe uses the migration's MySQL spelling, where the
+bare form is `ERROR 1064` on the reserved word `key`.
+
+**The referral, repointed rather than deleted** (maintainer ruling, 2026-08-22).
+All three conflict messages told the operator to "run `os migrate plan`" as an
+alternative way to list the blocking rows, and that instruction was false: they
+now name `os migrate duplicates`, which answers it. The six doc comments that
+state the same referral as part of the ADR-0120 D4 disposition are updated with
+them.
+
+**Nothing about a migration's behaviour changes.** No tightening is armed,
+deferred or altered, and `os migrate plan`'s drift contract is untouched. The
+pre-flight only makes the refusal's evidence readable one command before the
+restart — from a command that boots read-only and writes nothing, which is
+pinned logically (schema plus every row, ordered) rather than by a file hash: a
+raw hash over a SQLite file moves on any read-write open and would accuse this
+command of mutating the install it exists to describe.

content/docs/deployment/cli.mdx

@@ -677,7 +677,7 @@ where the data lives.
 | `os migrate value-shapes` | Scan stored reference and structured-JSON field values against the platform's value contract, and record the deployment's migration flag when clean |
 | `os migrate summary-nulls` | Backfill roll-up `count` / `sum` columns still stored as `NULL` on parent rows created before the insert-time seed. Repairs values; no flag, nothing depends on it having run |
 | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
-| `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions — a read-only inventory as JSON on stdout. Renumbers nothing and writes nothing at all; run it before the boot-time tenancy repair, which overwrites part of the evidence |
+| `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes nothing at all; run it before the boot-time tenancy repair, which overwrites part of the evidence |
 
 ```bash
 os migrate files-to-references # Dry run: full report, writes nothing
@@ -1011,6 +1011,57 @@ that could not be probed is listed under `skipped` with its reason, because
 a driver with no raw SQL seam (memory, MongoDB) fails the whole run with
 `error: "no_sql_seam"` rather than returning an empty inventory.
 
+##### The `kernel:ready` index pre-flight
+
+The report carries a second section, `runtimeIndexPreflight`, answering a
+different question: **will the next server start be able to finish tightening
+the platform's own unique indexes?**
+
+Three migrations run at `kernel:ready` on a serving boot (`os dev`, `os serve`,
+`os start`) and replace a declared UNIQUE index with the NULL-safe — and
+sometimes active-rows-only — form it was always meant to have:
+
+| Table | Index | What the tightening adds |
+| :--- | :--- | :--- |
+| `sys_metadata` | overlay `active` and `draft` | package-less overlays stop being NULL-distinct |
+| `sys_view_definition` | `idx_sys_view_def_active` | shared and environment-level views stop being NULL-distinct, and only active rows are constrained |
+| `sys_setting` | the declared row identity | tenant- and global-scope rows stop being NULL-distinct on `user_id` |
+
+Each is a **tightening**, so rows an installation already holds can block it.
+When that happens the migration refuses — the previous index stays in place, no
+row is touched, and the server keeps running — and reports it at `error` in the
+boot log. Until this section existed that log line was the only channel: these
+indexes are invisible to `os migrate plan` by construction, because the drift
+reconciler deliberately excludes runtime-managed indexes (otherwise the next
+boot would propose rebuilding away the guarantee it just created), and because
+each migration reuses the *declared* index's name, so the reconciler's slot for
+it reads as correctly filled whichever form is physically there.
+
+So the pre-flight lives here instead, on the command that already boots
+read-only and repairs nothing. It runs the migrations' own duplicate-listing
+queries — the exact statements the boot log prints — and reports one entry per
+index:
+
+| `status` | Meaning |
+| :--- | :--- |
+| `blocked` | Rows collide under the tightened key. `groups` lists each colliding key and how many rows hold it. The next serving boot will refuse this index |
+| `clear` | The probe ran and nothing collides |
+| `table-absent` | The table is not installed here. `sys_setting`, for instance, arrives with the optional settings service |
+| `unreadable` | The probe could not run; `detail` carries the driver's message |
+
+`summary.runtimeIndexesBlocked` and `summary.runtimeIndexBlockingRows` are the
+same finding counted at the head of the document.
+
+Read `blocked` as **work to do before the restart, not damage**: nothing is
+lost while an index stays untightened, but the guarantee it carries is not in
+force until the listed rows are resolved — and only an operator can decide which
+of two colliding rows survives, which is why the platform refuses rather than
+picking one.
+
+`--object` does not narrow this section. It is a fixed set of platform indexes
+rather than a slice of your registry, and `filter` describes the object scan
+only.
+
 ### Scaffolding
 
 | Command | Alias | Description |

docs/qa/platform-checklist/areas/cli.json

@@ -1047,7 +1047,7 @@
  "title": "os migrate duplicates: a read-only JSON inventory of identifiers minted across partitions — within-partition repeats excluded, nothing written, the live two-counter condition reported, and runnable BEFORE the #8686 repair destroys the evidence",
  "since": "v17",
  "status": "active",
- "revision": 2,
+ "revision": 3,
  "priority": "P1",
  "surface": "cli",
  "personas": ["operator (local shell, pre-repair audit)"],
@@ -1066,15 +1066,16 @@
  "boot the scratch app once with `os dev -d file:/tmp/<run>/dup.db` so the base schema exists; stop it",
  "seed via direct SQL per the fixture recipe: the cross-partition duplicate, the within-partition repeat, an organizations row for '<org>', and the paired sequence counters",
  "md5sum the DB file; run `os migrate duplicates > report.json; echo $?`; md5sum again and byte-compare",
- "jq the report: .report/.reportVersion/.generatedAt/.database/.globalPartition/.filter/.counters/.scanned/.skipped/.duplicates/.liveConditions/.summary",
+ "jq the report: .report/.reportVersion/.generatedAt/.database/.globalPartition/.filter/.counters/.scanned/.skipped/.duplicates/.liveConditions/.runtimeIndexPreflight/.summary",
+ "seed the kernel:ready blocker too (#8725): two ACTIVE sys_view_definition rows with the SAME name and organization_id/owner both NULL — then jq .runtimeIndexPreflight and .summary.runtimeIndexesBlocked",
  "run `os migrate duplicates --object <the-object>` and `--object <an-object-with-no-findings>` — capture .filter in both payloads",
  "run `os migrate duplicates --database-url file:/tmp/<run>/dup.db` and confirm it reaches the same DB (the flag also honors OS_DATABASE_URL)",
  "negative: from the memory-driver scratch config run `os migrate duplicates; echo $?` and capture the refusal payload",
  "stderr/stdout split: confirm report.json parses as ONE JSON document — the boot's own log lines must have gone to stderr (#6217)"
  ],
  "acceptance": [
  {
- "clause": "the report is the declared machine-readable contract on stdout: report 'duplicate-identifiers', reportVersion 1, generatedAt, database, globalPartition, filter, counters {table, status read|absent}, scanned[], skipped[], duplicates[], liveConditions[], summary — and each duplicate carries object/field/value/holderCount/partitions plus per-holder id/organization/partition/createdAt (createdAt null when the object has no such column, never a failed probe)",
+ "clause": "the report is the declared machine-readable contract on stdout: report 'duplicate-identifiers', reportVersion 2, generatedAt, database, globalPartition, filter, counters {table, status read|absent}, scanned[], skipped[], duplicates[], liveConditions[], runtimeIndexPreflight[], summary {…, runtimeIndexesBlocked, runtimeIndexBlockingRows} — and each duplicate carries object/field/value/holderCount/partitions plus per-holder id/organization/partition/createdAt (createdAt null when the object has no such column, never a failed probe)",
  "oracle": "log",
  "verify": "jq walks every declared key of the seeded run's payload; shape pinned by duplicates.contract.test.ts — cite its pass for the full-shape guarantee, drive the CLI for the instance",
  "evidence": "report.json + the jq walk"
@@ -1091,6 +1092,12 @@
  "verify": "the two md5sums match; duplicates.pre-repair.test.ts pins the same invariant down to _objectstack_sequences",
  "evidence": "the md5 pair"
  },
+ {
+ "clause": "the kernel:ready pre-flight (#8725) reports the RUNTIME-migration class os migrate plan cannot see: one entry per index the three kernel:ready migrations tighten (four — the overlay migration owns two), each blocked|clear|table-absent|unreadable, a blocked one naming every colliding key group and its row count",
+ "oracle": "log",
+ "verify": ".runtimeIndexPreflight names idx_sys_view_def_active as blocked with the seeded view name, organization_id_key '__global__' and owner_key '' — and the SAME database run through `os migrate plan` mentions neither the index nor the view name (the matched control: the declared-index duplicate above IS reported by plan, this one is not)",
+ "evidence": "the pre-flight section beside the plan output for one database"
+ },
  {
  "clause": "the live condition (#8928 point 4) fires exactly when a __global__ counter sits beside an org-scoped counter for the same object/field — reported as a prediction with globalLastValue and the per-org counters, and counters.status says where it was read from ('absent' still yields a complete duplicates inventory)",
  "oracle": "log",

packages/cli/src/commands/migrate/duplicates.contract.test.ts

@@ -21,7 +21,9 @@ import { tmpdir } from 'node:os';
 import { join } from 'node:path';
 import { SqlDriver } from '@objectstack/driver-sql';
 import {
+ collectRuntimeIndexPreflight,
  normalizeRows,
+ runtimeIndexProbes,
  GLOBAL_TENANT,
  ORGANIZATION_FIELD,
  SEQUENCES_TABLE,
@@ -120,7 +122,7 @@ afterAll(async () => {
  try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ }
 });
 
-const collect = (objectFilter?: string) =>
+const collect = async (objectFilter?: string) =>
  collectDuplicateIdentifierReport({
  exec,
  normalize: normalizeRows,
@@ -131,14 +133,31 @@ const collect = (objectFilter?: string) =>
  sequencesTable: SEQUENCES_TABLE,
  client: 'better-sqlite3',
  now: () => new Date('2026-08-17T12:00:00.000Z'),
+ // The real pre-flight against the real fixture — never a stand-in. This
+ // database has none of the four platform tables, so every entry is
+ // `table-absent`, and the `blocked` shape is pinned in its own test below
+ // over a database that really carries the damage.
+ runtimeIndexPreflight: await collectRuntimeIndexPreflight(exec, { client: 'better-sqlite3' }),
  ...(objectFilter ? { objectFilter } : {}),
  });
 
+/**
+ * The four probes, as `@objectstack/metadata-protocol` declares them.
+ *
+ * Read from the producer rather than restated here: the descriptor (table,
+ * index name, key parts, row scope, the two statements) is the migration's own
+ * definition of its key, and a second copy in this file would be a second
+ * definition to keep in step. What this file pins is that the report carries
+ * that descriptor through UNCHANGED, plus a status and its groups.
+ */
+const PROBES = runtimeIndexProbes({ client: 'better-sqlite3' });
+
 describe('#8928 os migrate duplicates — the report document', () => {
  it('is exactly this shape, whole', async () => {
  expect(await collect()).toEqual({
  report: 'duplicate-identifiers',
- reportVersion: 1,
+ // #8725 added `runtimeIndexPreflight` and its two summary counters.
+ reportVersion: 2,
  generatedAt: '2026-08-17T12:00:00.000Z',
  database: 'better-sqlite3 (fixture)',
  globalPartition: '__global__',
@@ -212,16 +231,40 @@ describe('#8928 os migrate duplicates — the report document', () => {
  organizationCounters: [{ organization: 'org_x', lastValue: 4 }],
  },
  ],
+ // One entry per index the three `kernel:ready` migrations tighten — FOUR,
+ // because the overlay migration builds one per state and either can be
+ // blocked on its own. Present whatever the outcome: an index left out
+ // would make "nothing blocks it" and "it was never probed" the same
+ // absence, which is the rule this command already applies to `skipped`.
+ runtimeIndexPreflight: PROBES.map((probe) => ({
+ ...probe,
+ status: 'table-absent',
+ groups: [],
+ })),
  summary: {
  objectsScanned: 2,
  fieldsScanned: 3,
  duplicateValues: 3,
  duplicateRows: 6,
  liveConditions: 1,
+ runtimeIndexesBlocked: 0,
+ runtimeIndexBlockingRows: 0,
  },
  });
  });
 
+ it('names the four kernel:ready indexes, and the migration behind each', async () => {
+ const report = await collect();
+ expect(
+ report.runtimeIndexPreflight.map((p) => `${p.migration}:${p.table}:${p.index}`),
+ ).toEqual([
+ 'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_active',
+ 'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_draft',
+ 'ensureViewDefinitionActiveIndex:sys_view_definition:idx_sys_view_def_active',
+ 'ensureSysSettingIdentityIndex:sys_setting:uniq_sys_setting_organization_id_namespace_key_scope_user_id',
+ ]);
+ });
+
  it('does NOT report a value repeated inside ONE partition — the narrow ruled definition', async () => {
  // `REF-1` is held twice, both times by `org_x`. That is a repeat the
  // partitioned unique index already refuses; reporting it would be the wider
@@ -248,6 +291,7 @@ describe('#8928 os migrate duplicates — the report document', () => {
  organizationField: ORGANIZATION_FIELD,
  sequencesTable: SEQUENCES_TABLE,
  client: 'better-sqlite3',
+ runtimeIndexPreflight: [],
  });
  expect(report.skipped).toEqual([
  expect.objectContaining({