build: merge driver for generator-owned spec artifacts (#4675)

.changeset/spec-generated-merge-driver.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/spec": patch
+---
+
+build: 为生成物加 `merge=os-regen` 合并驱动,把「集合运算被打成文本冲突」的返工消掉 (#4675)
+
+`packages/spec` 的生成物是排序数组与追加式登记表。两个 PR 各增删几行,语义上是集合并与集合差、完全可组合,git 却按三路文本合并报成需要人工解决的冲突 —— 2026-08-02 一个下午实测四次合并、九处冲突,**没有一次是真正的语义冲突**,每次的正确解法都是「丢掉两边、重新生成、重跑门禁」。
+
+`.gitattributes` 现在把这些路径交给 `scripts/git-merge-regen.mjs`。
+
+**驱动不做重算。** git 是在合并**过程中**按索引顺序调用 merge driver 的,那一刻工作区里还是合并前的源码:`packages/spec/spec-changes.json` 排在 `packages/spec/src/...` 之前,所以在驱动里跑生成器会读到缺了对方那半边改动的 `migrations/registry.ts`,写出一个自信而错误的产物 —— 比它取代的那个冲突更糟,因为冲突标记是可见的错误,而看起来合理的生成文件不是。改为**推迟**:驱动解析路径(不做文本合并、不留标记)并记入 `$GIT_DIR/os-regen-pending`,`pre-commit` 在产物重新生成之前拒绝提交。重算因此发生在合并后的完整树上 —— 唯一正确的时刻。
+
+`check:generated --fix` 现在在 `dist` 比 `src` 旧时**拒绝**运行 `gen:api-surface`,而不再只是警告。陈旧 dist 下该生成器不会失败,它会写出一份缺失了上次构建以来所有新导出的、看似合理的 surface,并让 `gen:docs` 顺手为这个缺口棘轮一条基线豁免(#4687 实际发生过,只靠与 `main` 对比生成物才发现)。`--fix` 是唯一会**写入**的路径,所以是这个陷阱唯一不可幸存的地方。
+
+只减不增的棘轮(`docs-import-surface.baseline.json`、`dual-source-exports.baseline.json`)与手写登记表刻意排除在外:重算一个只减不增的棘轮可能**放宽**它,等于把一条新豁免当作合并噪音洗进来。这些冲突仍然留给人看,逐条理由见 `scripts/regen-artifacts.mjs` 的 `NOT_DRIVER_MANAGED`。
+
+驱动按 clone 注册(`pnpm install` 经 `prepare` 完成)。没注册的 clone 回退到 git 默认文本合并 —— 即 #4675 之前的行为,不是故障。`pnpm check:merge-driver` 双向核对 `.gitattributes` 与该表,并对真实 git 做端到端验证。

.gitattributes

@@ -0,0 +1,37 @@
+# Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+#
+# Merge semantics for generator-owned artifacts (#4675).
+#
+# These files are sorted arrays and append-only ledgers derived from source. When
+# two PRs each add or drop a few lines the result is a set union — fully
+# composable — but a three-way TEXT merge reports it as a conflict a human must
+# resolve by hand. The correct resolution is always the same: discard both sides
+# and re-run the generator. `authorable-surface.json` alone is a 8k-line sorted
+# array, so any two PRs landing near each other collide.
+#
+# `merge=os-regen` hands those paths to `scripts/git-merge-regen.mjs`, which does
+# NOT text-merge them. See that file for why it also does not regenerate them
+# in place (git runs merge drivers BEFORE the sources are merged, so anything
+# computed there describes a half-merged tree).
+#
+# The driver is registered per clone by `scripts/setup-git-hooks.mjs`, which
+# `pnpm install` runs. A clone WITHOUT it registered falls back to git's default
+# text merge — i.e. exactly today's behaviour — so committing this file cannot
+# regress anyone.
+#
+# The single source of truth for this list is `scripts/regen-artifacts.mjs`;
+# `node scripts/git-merge-regen.mjs --self-test` reconciles the two in both
+# directions. Add a path there, not only here.
+#
+# Deliberately absent: docs-import-surface.baseline.json and
+# dual-source-exports.baseline.json (shrink-only ratchets — recomputing can
+# WIDEN them), variant-docs.json and the migrations/conversions registries
+# (hand-written). Those conflicts are for a human. See NOT_DRIVER_MANAGED.
+
+packages/spec/spec-changes.json merge=os-regen
+packages/spec/authorable-surface.json merge=os-regen
+packages/spec/json-schema.manifest.json merge=os-regen
+packages/spec/api-surface.json merge=os-regen
+packages/spec/api-surface-signatures.json merge=os-regen
+docs/protocol-upgrade-guide.md merge=os-regen
+content/docs/references/** merge=os-regen

.githooks/pre-commit

@@ -0,0 +1,11 @@
+#!/bin/sh
+# Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+#
+# Registered by `scripts/setup-git-hooks.mjs` via `core.hooksPath=.githooks`,
+# which `pnpm install` runs. Cheap by construction: with no pending marker it
+# exits before doing any work, which is every commit that did not just merge a
+# generator-owned artifact (#4675).
+
+if [ -z "$OS_SKIP_REGEN_CHECK" ]; then
+ node "$(git rev-parse --show-toplevel)/scripts/check-regen-pending.mjs" || exit 1
+fi

AGENTS.md

@@ -228,6 +228,42 @@ Even inside your own worktree, operate defensively:
  commit itself — that second CI round is where joint breakage surfaces, and
  the guards in `scripts/check-*.mjs` exist largely because this class of
  breakage is invisible to `git merge`.
+11. **Generated artifacts don't text-merge — a driver defers them and
+ `pre-commit` collects the debt.** §10's "never trust git's textual merge of a
+ generated file" is now mechanical (#4675). `.gitattributes` routes the
+ generator-owned artifacts (`spec-changes.json`, `authorable-surface.json`,
+ `api-surface*.json`, `json-schema.manifest.json`,
+ `docs/protocol-upgrade-guide.md`, `content/docs/references/**`) to
+ `merge=os-regen`, so a merge that used to stop on conflicts across all of
+ them now stops only on the hand-written files that actually need you.
+
+ The driver does **not** regenerate. Git runs merge drivers *while* it merges,
+ in index order, so the worktree still holds pre-merge sources — a generator
+ run there would describe a half-merged tree and write a confidently wrong
+ artifact, which is strictly worse than the conflict it replaced. Instead it
+ records each path in `$GIT_DIR/os-regen-pending`, and `pre-commit` refuses the
+ commit until those artifacts check clean. So the sequence after a merge is
+ unchanged from §9 — rebuild, then `check:generated --fix` — you just cannot
+ forget it.
+
+ Two things worth knowing:
+ - **Registration is per clone.** `pnpm install` does it (`prepare` →
+ `scripts/setup-git-hooks.mjs`). A clone where that never ran falls back to
+ git's default text merge — pre-#4675 behaviour, not breakage — so nothing
+ depends on every machine being set up.
+ - **The ratchets are deliberately excluded**
+ (`docs-import-surface.baseline.json`, `dual-source-exports.baseline.json`,
+ the hand-written `migrations`/`conversions` registries, `variant-docs.json`).
+ Recomputing a shrink-only ratchet can *widen* it, which would launder a new
+ exemption in as merge noise. Those conflicts are yours to read. See
+ `NOT_DRIVER_MANAGED` in `scripts/regen-artifacts.mjs` for why, per path.
+
+ Related: `check:generated --fix` now **refuses** to run `gen:api-surface` on a
+ stale `dist` rather than warning about it (§9's trap, made unsurvivable on the
+ one path that writes).
+
+ `pnpm check:merge-driver` reconciles `.gitattributes` against that table in
+ both directions and proves the driver end to end against real git.
 
 ---
 

package.json

@@ -15,6 +15,8 @@
  "typecheck": "turbo run typecheck",
  "clean": "turbo run clean && rm -rf dist",
  "setup": "pnpm install && pnpm --filter @objectstack/spec build",
+ "prepare": "node scripts/setup-git-hooks.mjs",
+ "check:merge-driver": "node scripts/git-merge-regen.mjs --self-test && node scripts/check-regen-pending.mjs --self-test",
  "version": "changeset version && node scripts/sync-protocol-version.mjs && node scripts/sync-template-versions.mjs",
  "release": "pnpm run build && bash scripts/build-console.sh && bash scripts/release-publish.sh",
  "docs:dev": "pnpm --filter @objectstack/docs dev",

packages/spec/scripts/check-generated.ts

@@ -30,6 +30,11 @@ import { readFileSync } from 'node:fs';
 import { dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
 
+// One staleness rule, shared with the merge driver's pre-commit half (#4675) —
+// two copies of "is dist older than src" would drift, and the direction they
+// drift in is the one that writes a wrong artifact.
+import { distIsStale } from '../../../scripts/check-regen-pending.mjs';
+
 const pkgRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
 
 /**
@@ -232,6 +237,24 @@ if (!fix) {
 console.log(`\n--fix: regenerating the ${stale.length} stale artifact(s). Review the diff before committing.\n`);
 let failed = 0;
 for (const s of stale) {
+ // The `readsDist` warning above is advice a reader can ignore; here it must
+ // become a refusal. `gen:api-surface` on a stale dist does not fail — it
+ // writes a plausible surface with every export added since the last build
+ // missing, and `gen:docs` then ratchets a baseline exemption in to cover the
+ // hole. That landed unnoticed on #4687 and was caught only by diffing the
+ // generated files against `main`. --fix is the one path that WRITES, so it is
+ // the one place the trap is unsurvivable: a visible conflict is recoverable,
+ // a confidently wrong artifact is not (#4675).
+ if (s.readsDist && distIsStale()) {
+ failed++;
+ console.log(` ✗ ${s.gen} — REFUSED`);
+ console.error(
+ ` packages/spec/dist is missing or older than packages/spec/src.\n`
+ + ` Regenerating now would write a surface describing a build that no longer exists.\n`
+ + ` pnpm --filter @objectstack/spec build && pnpm --filter @objectstack/spec ${s.gen}`,
+ );
+ continue;
+ }
  const { ok, output } = run(s.gen);
  console.log(` ${ok ? '✓' : '✗'} ${s.gen}`);
  if (!ok) {

scripts/check-regen-pending.mjs

@@ -0,0 +1,166 @@
+#!/usr/bin/env node
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * The other half of the `merge=os-regen` driver (#4675): make the deferred
+ * regeneration **mandatory** instead of remembered.
+ *
+ * The driver resolves generator-owned artifacts without text-merging them and
+ * records each one in `$GIT_DIR/os-regen-pending`. It cannot regenerate them
+ * itself — git runs merge drivers before the sources are merged, so anything
+ * computed there describes a half-merged tree (see `git-merge-regen.mjs`). This
+ * runs from `pre-commit`, where the merged tree finally exists, and refuses the
+ * commit while any pending artifact is still stale.
+ *
+ * It **verifies, then clears** — it does not regenerate. Blanket regeneration
+ * from a hook would rewrite artifacts whose staleness nobody saw, which is the
+ * signal-destroying behaviour `check:generated` already refuses for the same
+ * reason. And a marker cannot get stuck: the moment the artifacts check clean,
+ * whether you regenerated them or the merge simply did not change them, the
+ * marker is removed and the commit proceeds.
+ *
+ * ## The dist trap
+ *
+ * `gen:api-surface` reads the BUILT `dist/*.d.ts`. On a stale dist it does not
+ * fail — it emits a plausible surface missing every export added since the last
+ * build. So for `readsDist` artifacts this refuses to even run the gate unless
+ * the build is newer than the sources, because a phantom "breaking removal" has
+ * cost real triage time before (#4687, and the trap is recorded in AGENTS.md).
+ *
+ * Usage:
+ * node scripts/check-regen-pending.mjs # pre-commit
+ * node scripts/check-regen-pending.mjs --self-test # no repo state touched
+ */
+
+import { execFileSync, execSync } from 'node:child_process';
+import { existsSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+import { PENDING_MARKER, entryForPath } from './regen-artifacts.mjs';
+
+const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
+const SPEC_DIR = join(REPO_ROOT, 'packages/spec');
+
+/** Newest mtime under `dir` for files matching `pred`, or 0 when there are none. */
+function newestMtime(dir, pred, depth = 0) {
+ if (depth > 12 || !existsSync(dir)) return 0;
+ let newest = 0;
+ for (const e of readdirSync(dir, { withFileTypes: true })) {
+ if (e.name === 'node_modules' || e.name.startsWith('.')) continue;
+ const p = join(dir, e.name);
+ if (e.isDirectory()) newest = Math.max(newest, newestMtime(p, pred, depth + 1));
+ else if (pred(e.name)) newest = Math.max(newest, statSync(p).mtimeMs);
+ }
+ return newest;
+}
+
+/**
+ * Is `packages/spec/dist` older than the sources it claims to describe? Missing
+ * counts as stale. Deliberately conservative: a false "stale" costs a build, a
+ * false "fresh" costs a silently wrong artifact.
+ */
+export function distIsStale(specDir = SPEC_DIR) {
+ const dist = newestMtime(join(specDir, 'dist'), (n) => n.endsWith('.d.ts'));
+ if (!dist) return true;
+ return newestMtime(join(specDir, 'src'), (n) => n.endsWith('.ts')) > dist;
+}
+
+function markerPath() {
+ const gitDir = execFileSync('git', ['rev-parse', '--absolute-git-dir'], { encoding: 'utf8' }).trim();
+ return join(gitDir, PENDING_MARKER);
+}
+
+function readPending(marker) {
+ if (!existsSync(marker)) return [];
+ return [...new Set(readFileSync(marker, 'utf8').split('\n').map((l) => l.trim()).filter(Boolean))];
+}
+
+function runCheck(script) {
+ try {
+ execSync(`pnpm -s ${script}`, { cwd: SPEC_DIR, stdio: ['ignore', 'pipe', 'pipe'] });
+ return { ok: true, output: '' };
+ } catch (err) {
+ return { ok: false, output: `${err?.stdout?.toString() ?? ''}${err?.stderr?.toString() ?? ''}`.trim() };
+ }
+}
+
+function main() {
+ const marker = markerPath();
+ const pending = readPending(marker);
+ if (!pending.length) return 0;
+
+ const entries = pending.map((p) => ({ path: p, entry: entryForPath(p) })).filter((x) => x.entry);
+ const unknown = pending.filter((p) => !entryForPath(p));
+
+ console.error(
+ `\nos-regen: ${pending.length} generated artifact(s) were merged WITHOUT a text merge and must be `
+ + `regenerated from the merged tree before this commit.\n`,
+ );
+
+ // Group by gate: `gen:schema` owns two artifacts, so running it twice is waste.
+ const byCheck = new Map();
+ for (const { path, entry } of entries) {
+ const g = byCheck.get(entry.check) ?? { entry, paths: [] };
+ g.paths.push(path);
+ byCheck.set(entry.check, g);
+ }
+
+ let blocked = 0;
+ for (const [check, { entry, paths }] of byCheck) {
+ if (entry.readsDist && distIsStale()) {
+ blocked++;
+ console.error(
+ ` ✗ ${paths.join(', ')}\n`
+ + ` ${check} reads packages/spec/dist, which is older than src — NOT running it.\n`
+ + ` On a stale dist this gate reports phantom removals and the generator WRITES them.\n`
+ + ` pnpm --filter @objectstack/spec build && pnpm --filter @objectstack/spec ${entry.gen}`,
+ );
+ continue;
+ }
+ const { ok, output } = runCheck(check);
+ if (ok) {
+ console.error(` ✓ ${paths.join(', ')} — current`);
+ continue;
+ }
+ blocked++;
+ const detail = output.split('\n').filter(Boolean).slice(0, 3).map((l) => ` ${l}`).join('\n');
+ console.error(` ✗ ${paths.join(', ')} — stale\n${detail ? `${detail}\n` : ''}`
+ + ` pnpm --filter @objectstack/spec ${entry.gen}`);
+ }
+
+ for (const p of unknown) {
+ blocked++;
+ console.error(` ✗ ${p} — recorded as pending but absent from scripts/regen-artifacts.mjs (cannot verify)`);
+ }
+
+ if (blocked) {
+ console.error(
+ `\nRegenerate the ${blocked} stale artifact(s) above, \`git add\` them, and commit again.\n`
+ + ' This check clears itself the moment they are current — nothing to reset by hand.\n'
+ + ' Bypass with --no-verify only if you intend CI to catch it: every one of these has a\n'
+ + ' required gate on the PR.\n',
+ );
+ return 1;
+ }
+
+ rmSync(marker, { force: true });
+ console.error('os-regen: all deferred artifacts are current — marker cleared.\n');
+ return 0;
+}
+
+// `check:generated --fix` imports `distIsStale` from here, so nothing may run on
+// import — only when this file IS the entry point.
+const invokedDirectly = process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url);
+
+if (invokedDirectly) {
+ if (process.argv.includes('--self-test')) {
+ // Touches no repo state: the interesting logic is the staleness rule, and its
+ // dangerous direction is "says fresh when stale".
+ const ok = distIsStale(join(REPO_ROOT, 'scripts')) === true;
+ console.log(`${ok ? '✓' : '✗'} a directory with no dist/ reads as STALE (conservative default)`);
+ console.log(ok ? '\n✓ check-regen-pending self-test passed.' : '\n✗ self-test failed.');
+ process.exit(ok ? 0 : 1);
+ }
+ process.exit(main());
+}