From 8c64eb5917b14788004c8193481bfd55a935a530 Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Sun, 23 Aug 2026 22:27:44 +0800 Subject: [PATCH 1/2] =?UTF-8?q?feat(seed):=20declared=20pointer-pair=20res?= =?UTF-8?q?olution=20=E2=80=94=20seeds=20can=20address=20an=20ActivityPoin?= =?UTF-8?q?ter=20(#11339)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A text field may declare `referenceVia: ''`, marking it as the id half of a polymorphic pointer pair (ADR-0052 §5) whose target object the sibling column names per row. The seed loader derives a per-row reference from the pair and routes it through the same resolution machinery static lookups use — externalId probes, in-memory map, pass-2 deferral — and refuses an unresolvable or un-addressable pointer loudly instead of storing the literal natural key as a row that attaches to nothing. sys_activity adopts the carrier on both pairs (record_id via object_name, source_id via source_object), so a packaged app's seed can ship timeline rows the shipped console filter { object_name, record_id } actually finds. Authoring contradictions are refused at parse (text-only, exclusive with reference) and at ObjectSchema.create (sibling must be declared). Undeclared text columns keep today's verbatim behavior. Co-Authored-By: Claude Opus 5 --- .changeset/pointer-pair-seed-resolution.md | 9 + content/docs/references/data/field.mdx | 1 + .../src/seed-loader-pointer-pair.test.ts | 414 ++++++++++++++++++ packages/metadata-protocol/src/seed-loader.ts | 117 ++++- .../src/objects/sys-activity.object.ts | 9 + packages/spec/authorable-surface/data.json | 1 + packages/spec/liveness/field.json | 6 + packages/spec/liveness/state-counts.md | 4 +- packages/spec/src/data/field.test.ts | 97 ++++ packages/spec/src/data/field.zod.ts | 55 +++ packages/spec/src/data/object.zod.ts | 46 ++ 11 files changed, 756 insertions(+), 3 deletions(-) create mode 100644 .changeset/pointer-pair-seed-resolution.md create mode 100644 packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts diff --git a/.changeset/pointer-pair-seed-resolution.md b/.changeset/pointer-pair-seed-resolution.md new file mode 100644 index 0000000000..c0ea6591e6 --- /dev/null +++ b/.changeset/pointer-pair-seed-resolution.md @@ -0,0 +1,9 @@ +--- +"@objectstack/spec": minor +"@objectstack/metadata-protocol": minor +"@objectstack/plugin-audit": minor +--- + +Seeds can now address an ActivityPointer (#11339, ADR-0052 §5): a `text` field may declare `referenceVia: ''`, marking it as the id half of a polymorphic pointer pair whose target object the sibling column names per row (`sys_activity.record_id` via `object_name`, `source_id` via `source_object` — both now declared). The seed loader resolves such pointers as natural keys against the object each row names — the same externalId probes, in-memory map and pass-2 deferral static lookup references use — so a packaged app's seed can ship timeline rows that actually attach to their records, and the shipped console filter `{ object_name, record_id }` matches them. + +The accept/reject contract changes with it, deliberately: an unresolvable pointer on a DECLARED pair is now a loud, counted failure (`success: false`, `error`-level log, record dropped when no pass 2 can heal it) instead of the old silent verbatim store — a row that rendered on no timeline and matched no filter. Undeclared text columns are untouched: only `referenceVia` opts a pair in. Authoring contradictions are refused at parse time (`referenceVia` is text-only and mutually exclusive with `reference`) and at `ObjectSchema.create` (the sibling must be a declared field). Internal-id-shaped values still pass through verbatim, so seeds wiring real ids keep working. diff --git a/content/docs/references/data/field.mdx b/content/docs/references/data/field.mdx index ad24ab5b4b..71737f1cea 100644 --- a/content/docs/references/data/field.mdx +++ b/content/docs/references/data/field.mdx @@ -76,6 +76,7 @@ const result = CurrencyConfigSchema.parse(data); | **maxSize** | `integer` | optional | Maximum permitted file size in BYTES for media fields. Enforced on write against the stored file size, not just checked in the browser. | | **options** | `{ label: string; value: string; color?: string; default?: boolean; … }[]` | optional | Static options for select/multiselect | | **reference** | `string` | optional | Target object name (snake_case) for lookup/master_detail fields. Required for relationship types. Used by $expand to resolve foreign key IDs into full objects. | +| **referenceVia** | `string` | optional | Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): the value is a record id of the object named by the SIBLING FIELD this key names — e.g. `record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same object holding an object machine name. Text fields only; mutually exclusive with `reference` (a static and a per-record target contradict). Enforced today at seed load: the value resolves as a natural key against the object the sibling column names, and an unresolvable pointer is refused loudly instead of stored verbatim. Adds no referential integrity or $expand behavior. | | **deleteBehavior** | `Enum<'set_null' \| 'cascade' \| 'restrict'>` | optional (default: `"set_null"`) | What happens if referenced record is deleted | | **inlineEdit** | `boolean \| Enum<'grid' \| 'form'>` | optional | Edit these child records inline within the parent's form (atomic master-detail). true = auto-pick grid/form by child shape; 'grid' = editable line-item grid; 'form' = list + per-row full form. | | **inlineTitle** | `string` | optional | Title for the inline master-detail grid | diff --git a/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts b/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts new file mode 100644 index 0000000000..0788722839 --- /dev/null +++ b/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts @@ -0,0 +1,414 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect, vi } from 'vitest'; +import { SeedLoaderService } from './seed-loader'; +import type { IDataEngine, IMetadataService } from '@objectstack/spec/contracts'; +import { assertEngineDeleteDispatch, assertEngineUpdateDispatch } from '@objectstack/metadata-core'; + +/** + * Declared pointer-pair resolution — #11339 (ADR-0052 §5, the ActivityPointer + * model). + * + * `sys_activity.record_id` is a plain `text` column whose value is a record id + * of the object the sibling `object_name` column names. Before #11339 the seed + * loader resolved natural keys only for `lookup`/`master_detail`/`user` + * fields, so a packaged app's activity seed loaded "successfully" while + * storing the literal natural key — rows that attach to nothing: the shipped + * console timeline filters on `{ object_name, record_id }` with the target's + * REAL id and finds zero rows (measured downstream in + * objectstack-ai/hotcrm#1258). + * + * The fix is a declared carrier: a `text` field carrying + * `referenceVia: ''` derives a per-ROW reference whose target object + * is read from the sibling column, and flows through the SAME resolution + * machinery as static references — same probes, same deferral, same loud + * refusal. These tests pin both halves of the contract change: + * - the ACCEPT half: natural keys now resolve to internal ids (per row, + * in-memory and DB probes, pass-2 deferral for order independence); + * - the REFUSE half: an unresolvable or un-addressable pointer is a loud, + * counted failure — never the silently stored literal it used to be. + */ + +function createLogger() { + return { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() }; +} + +/** Same faithful engine shape as seed-loader-engine-schema-fallback.test.ts: + * where-filtering find, id-dispatch-asserting update/delete, and a + * `getSchema` backed by the schema map (the ObjectQL registry stand-in). */ +function createFaithfulEngine(schemas: Record) { + const store: Record = {}; + let idCounter = 0; + + const engine = { + find: vi.fn(async (objectName: string, query?: any) => { + let records = store[objectName] || []; + if (query?.where) { + records = records.filter((r) => + Object.entries(query.where).every(([k, v]) => { + if (k.startsWith('$')) throw new Error(`fake driver: unsupported operator ${k}`); + return r[k] === v; + }), + ); + } + if (typeof query?.limit === 'number') records = records.slice(0, query.limit); + return records; + }), + findOne: vi.fn(async (objectName: string, query?: any) => { + const rows = await (engine.find as any)(objectName, { ...query, limit: 1 }); + return rows[0] ?? null; + }), + insert: vi.fn(async (objectName: string, data: any) => { + if (!store[objectName]) store[objectName] = []; + if (Array.isArray(data)) { + const records = data.map((d) => ({ id: `gen-${++idCounter}`, ...d })); + store[objectName].push(...records); + return records; + } + const record = { id: `gen-${++idCounter}`, ...data }; + store[objectName].push(record); + return record; + }), + update: vi.fn(async (objectName: string, data: any) => { + assertEngineUpdateDispatch(data, undefined); + const records = store[objectName] || []; + const idx = records.findIndex((r) => r.id === data.id); + if (idx >= 0) { + records[idx] = { ...records[idx], ...data }; + return records[idx]; + } + return data; + }), + delete: vi.fn(async (_objectName: string, options?: any) => { + assertEngineDeleteDispatch(options); + return { deleted: 1 }; + }), + count: vi.fn(async (objectName: string) => (store[objectName] || []).length), + aggregate: vi.fn(async () => []), + getSchema: vi.fn((objectName: string) => schemas[objectName]), + } as unknown as IDataEngine & { getSchema: ReturnType }; + + return { engine, store }; +} + +/** Objects known only to the engine registry — the marketplace-install path + * the downstream measurement ran on (metadata service knows nothing). */ +function createEmptyMetadata(): IMetadataService { + return { + getObject: vi.fn(async () => undefined), + listObjects: vi.fn(async () => []), + register: vi.fn(async () => {}), + get: vi.fn(async () => undefined), + list: vi.fn(async () => []), + unregister: vi.fn(async () => {}), + exists: vi.fn(async () => false), + listNames: vi.fn(async () => []), + } as unknown as IMetadataService; +} + +/** The measured shape: a business target plus the sys_activity pointer pairs + * exactly as plugin-audit declares them after #11339. */ +const SCHEMAS: Record = { + crm_lead: { + name: 'crm_lead', + fields: { + name: { type: 'text', required: true }, + status: { type: 'text' }, + }, + }, + sys_activity: { + name: 'sys_activity', + fields: { + type: { type: 'select' }, + summary: { type: 'text', required: true }, + object_name: { type: 'text' }, + record_id: { type: 'text', referenceVia: 'object_name' }, + record_label: { type: 'text' }, + source_object: { type: 'text' }, + source_id: { type: 'text', referenceVia: 'source_object' }, + }, + }, + // The counter-example: same column names, NO declaration — the loader must + // keep treating these as plain text (stored verbatim), because resolving + // undeclared pairs would rewrite data on every object that happens to share + // the idiom's spelling. + sys_audit_like: { + name: 'sys_audit_like', + fields: { + summary: { type: 'text', required: true }, + object_name: { type: 'text' }, + record_id: { type: 'text' }, + }, + }, +}; + +const CONFIG = { + dryRun: false, + haltOnError: false, + multiPass: true, + defaultMode: 'upsert', + batchSize: 1000, + transaction: false, +} as any; + +const LEAD_SEED = { + object: 'crm_lead', + externalId: 'name', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records: [{ name: 'Lisa Thompson', status: 'qualified' }], +}; + +function activitySeed(records: Array>) { + return { + object: 'sys_activity', + externalId: 'summary', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records, + }; +} + +function newService(schemas: Record = SCHEMAS) { + const { engine, store } = createFaithfulEngine(schemas); + const logger = createLogger(); + const service = new SeedLoaderService(engine, createEmptyMetadata(), logger); + return { service, engine, store, logger }; +} + +describe('seed pointer-pair resolution (#11339 — referenceVia)', () => { + it('resolves record_id through the object object_name names, so the console filter shape matches', async () => { + const { service, store } = newService(); + + const result = await service.load({ + seeds: [ + LEAD_SEED, + activitySeed([ + { type: 'completed', summary: 'Discovery call', object_name: 'crm_lead', record_id: 'Lisa Thompson' }, + ]), + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + const leadId = store.crm_lead.find((r) => r.name === 'Lisa Thompson')!.id; + // The exact filter the shipped console bundle issues — the downstream + // measurement's failing read, now the passing one: + const attached = store.sys_activity.filter( + (r) => r.object_name === 'crm_lead' && r.record_id === leadId, + ); + expect(attached).toHaveLength(1); + // …and the measured failure shape is gone: no row stores the literal key. + expect(store.sys_activity.filter((r) => r.record_id === 'Lisa Thompson')).toHaveLength(0); + }); + + it('resolves each row against ITS OWN sibling value, and the source pair independently of the regarding pair', async () => { + const { service, store } = newService({ + ...SCHEMAS, + crm_case: { name: 'crm_case', fields: { name: { type: 'text', required: true } } }, + sys_email: { name: 'sys_email', fields: { name: { type: 'text', required: true } } }, + }); + + const result = await service.load({ + seeds: [ + LEAD_SEED, + { + object: 'crm_case', + externalId: 'name', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records: [{ name: 'Broken widget' }], + }, + { + object: 'sys_email', + externalId: 'name', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records: [{ name: 'Re: pricing' }], + }, + activitySeed([ + // Row 1 points at the lead; its source drills to the email row. + { + type: 'completed', summary: 'Email follow-up', + object_name: 'crm_lead', record_id: 'Lisa Thompson', + source_object: 'sys_email', source_id: 'Re: pricing', + }, + // Row 2 points at a DIFFERENT object — per-row targets, one dataset. + { type: 'created', summary: 'Case opened', object_name: 'crm_case', record_id: 'Broken widget' }, + ]), + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + const leadId = store.crm_lead[0].id; + const caseId = store.crm_case[0].id; + const emailId = store.sys_email[0].id; + const row1 = store.sys_activity.find((r) => r.summary === 'Email follow-up')!; + const row2 = store.sys_activity.find((r) => r.summary === 'Case opened')!; + expect(row1.record_id).toBe(leadId); + expect(row1.source_id).toBe(emailId); + expect(row2.record_id).toBe(caseId); + }); + + it('resolves through the target DATASET\'s declared externalId, like static references do', async () => { + const { service, store } = newService({ + ...SCHEMAS, + crm_lead: { + name: 'crm_lead', + fields: { name: { type: 'text' }, email: { type: 'text', required: true } }, + }, + }); + + const result = await service.load({ + seeds: [ + { + object: 'crm_lead', + externalId: 'email', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records: [{ name: 'Lisa Thompson', email: 'lisa@example.com' }], + }, + activitySeed([ + { type: 'completed', summary: 'Call', object_name: 'crm_lead', record_id: 'lisa@example.com' }, + ]), + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + expect(store.sys_activity[0].record_id).toBe(store.crm_lead[0].id); + }); + + it('defers to pass 2 when the activity dataset loads before its target — order independence', async () => { + const { service, store, engine } = newService(); + + // sys_activity FIRST: no static dependency edge exists (the target is a + // per-row fact), so topological order cannot save this — pass 2 must. + const result = await service.load({ + seeds: [ + activitySeed([ + { type: 'completed', summary: 'Discovery call', object_name: 'crm_lead', record_id: 'Lisa Thompson' }, + ]), + LEAD_SEED, + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + const leadId = store.crm_lead.find((r) => r.name === 'Lisa Thompson')!.id; + expect(store.sys_activity[0].record_id).toBe(leadId); + // It really went through the back-fill write, not through luck of ordering. + expect((engine.update as any).mock.calls.length).toBeGreaterThan(0); + const activityResult = result.results.find((r: any) => r.object === 'sys_activity')!; + expect(activityResult.referencesResolved).toBeGreaterThan(0); + }); + + it('REFUSES an unresolvable pointer loudly instead of storing the literal (the measured silent failure)', async () => { + const { service, store, logger } = newService(); + + const result = await service.load({ + seeds: [ + LEAD_SEED, + activitySeed([ + { type: 'completed', summary: 'Ghost call', object_name: 'crm_lead', record_id: 'No Such Lead' }, + ]), + ] as any, + config: CONFIG, + }); + + // The contract change this card ships: this load can no longer read as a + // clean success — before #11339 it was `success: true` with the literal + // stored, at every gate green. + expect(result.success).toBe(false); + expect(result.errors.some((e: any) => e.field === 'record_id' && String(e.message).includes('crm_lead'))).toBe(true); + expect(logger.error).toHaveBeenCalled(); + // Never stored verbatim: the row (written in pass 1, pointer deferred) + // carries NO record_id rather than the unresolvable literal. + const row = store.sys_activity.find((r) => r.summary === 'Ghost call')!; + expect(row.record_id ?? null).toBeNull(); + expect(store.sys_activity.filter((r) => r.record_id === 'No Such Lead')).toHaveLength(0); + }); + + it('REFUSES an un-addressable pointer (id half authored, type half empty) — the record is not seeded', async () => { + const { service, store, logger } = newService(); + + const result = await service.load({ + seeds: [ + activitySeed([ + { type: 'completed', summary: 'Orphan pointer', record_id: 'Lisa Thompson' }, + ]), + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(false); + const err = result.errors.find((e: any) => e.field === 'record_id'); + expect(err).toBeDefined(); + // The message names the pair field so the author knows the one-line fix. + expect(String(err!.message)).toContain('object_name'); + expect(logger.error).toHaveBeenCalled(); + expect(store.sys_activity ?? []).toHaveLength(0); + const activityResult = result.results.find((r: any) => r.object === 'sys_activity')!; + expect(activityResult.errored).toBe(1); + }); + + it('keeps an internal-id-shaped value verbatim (advanced seeds may wire real ids)', async () => { + const { service, store } = newService(); + const uuid = '123e4567-e89b-42d3-a456-426614174000'; + + const result = await service.load({ + seeds: [ + activitySeed([ + { type: 'completed', summary: 'Wired by id', object_name: 'crm_lead', record_id: uuid }, + ]), + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + expect(store.sys_activity[0].record_id).toBe(uuid); + }); + + it('leaves UNDECLARED pairs alone: same column names without referenceVia still store verbatim', async () => { + const { service, store } = newService(); + + const result = await service.load({ + seeds: [ + { + object: 'sys_audit_like', + externalId: 'summary', + mode: 'upsert', + env: ['prod', 'dev', 'test'], + records: [{ summary: 'Plain text row', object_name: 'crm_lead', record_id: 'Lisa Thompson' }], + }, + ] as any, + config: CONFIG, + }); + + expect(result.success).toBe(true); + // Undeclared = untouched: the loader must not invent pointer semantics + // from column spellings — only the declared carrier opts an object in. + expect(store.sys_audit_like[0].record_id).toBe('Lisa Thompson'); + }); + + it('dry-run reports the would-fail pointer without writing or logging at error', async () => { + const { service, store, logger } = newService(); + + const result = await service.load({ + seeds: [ + activitySeed([ + { type: 'completed', summary: 'Orphan pointer', record_id: 'Lisa Thompson' }, + { type: 'completed', summary: 'Ghost target', object_name: 'crm_lead', record_id: 'No Such Lead' }, + ]), + ] as any, + config: { ...CONFIG, dryRun: true }, + }); + + // Both defects are visible in the report the caller reads… + expect(result.errors.filter((e: any) => e.field === 'record_id').length).toBe(2); + // …and nothing was written or shouted about a simulated outcome (#4997). + expect(store.sys_activity ?? []).toHaveLength(0); + expect(logger.error).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/metadata-protocol/src/seed-loader.ts b/packages/metadata-protocol/src/seed-loader.ts index 519df899e2..66b628b229 100644 --- a/packages/metadata-protocol/src/seed-loader.ts +++ b/packages/metadata-protocol/src/seed-loader.ts @@ -219,6 +219,11 @@ function isEnvScopedDataset(dataset: Seed): boolean { * - Automatic lookup/master_detail reference resolution via externalId * (in-memory for records seeded this load, DB probe by the target * dataset's declared externalId otherwise) + * - Declared pointer-pair resolution (#11339, ADR-0052 §5 ActivityPointer): + * a `text` field carrying `referenceVia` resolves per ROW against the + * object its sibling column names (`sys_activity.record_id` through + * `object_name`), by the same externalId rules — and an unresolvable or + * un-addressable pointer is refused loudly, never stored verbatim * - Topological dependency ordering (parents before children) * - Multi-pass loading for circular references * - Dry-run validation mode @@ -273,6 +278,27 @@ export class SeedLoaderService implements ISeedLoaderService { * the very column this answer is about. */ private declaresNameColumnCache = new Map(); + /** + * [#11339] Declared pointer pairs per seeded object — every `text` field + * whose definition carries `referenceVia` (ADR-0052 §5 ActivityPointer: + * `sys_activity.record_id` + `object_name`), mapped to the sibling field + * that names the target object PER ROW. Built once per {@link load} (step + * 4.5) from the same definition resolver the reference graph uses. + * + * An instance field for the same reason {@link fallbackOrgId} is one: the + * consumer ({@link loadDataset}'s per-record resolution loop) sits several + * parameters deep, and the map is a per-load constant. Reset per `load`. + */ + private pointerRefsByObject = new Map>(); + /** + * [#11339] The per-load `dataset.object → declared externalId` map — the + * SAME map step 4 threads into {@link buildReferenceMap} for static + * references. A pointer pair's target object is only known per ROW, so its + * `targetField` cannot be pre-resolved into a refMap entry; the resolution + * loop reads this instead, at the moment the sibling column names the + * target. Reset per `load`. + */ + private seedExternalIdByObject = new Map(); constructor(engine: IDataEngine, metadata: IMetadataService, logger: Logger) { this.engine = engine; @@ -347,6 +373,30 @@ export class SeedLoaderService implements ISeedLoaderService { ); const refMap = this.buildReferenceMap(graph, externalIdByObject); + // 4.5 [#11339] Collect declared pointer pairs (`referenceVia`) per seeded + // object. These are deliberately NOT part of the dependency graph or the + // refMap: the target object is named per ROW by the sibling column, so a + // pointer contributes no static ordering edge — same-load targets that + // happen to load later are healed by the ordinary pass-2 deferral, which + // is per record anyway. Only `text` fields participate: the spec refuses + // `referenceVia` on other types at authoring, and metadata at rest that + // predates that check must not have non-text columns resolved as ids. + this.pointerRefsByObject.clear(); + this.seedExternalIdByObject = externalIdByObject; + for (const dataset of orderedDatasets) { + if (this.pointerRefsByObject.has(dataset.object)) continue; + const objDef = await this.resolveObjectDefinition(dataset.object); + const fields = objDef?.fields as Record | undefined; + if (!fields) continue; + const pairs: Array<{ field: string; objectField: string }> = []; + for (const [fieldName, fieldDef] of Object.entries(fields)) { + if (fieldDef?.type === 'text' && typeof fieldDef.referenceVia === 'string' && fieldDef.referenceVia.length > 0) { + pairs.push({ field: fieldName, objectField: fieldDef.referenceVia }); + } + } + if (pairs.length > 0) this.pointerRefsByObject.set(dataset.object, pairs); + } + // 5. Pass 1: Insert/upsert records, resolving references const insertedRecords = new Map>(); // object → externalIdValue → internalId const deferredUpdates: DeferredUpdate[] = []; @@ -762,7 +812,72 @@ export class SeedLoaderService implements ISeedLoaderService { // Resolve references let unresolvedRefError = false; - for (const ref of objectRefs) { + + // [#11339] Derive this RECORD's pointer-pair references (ADR-0052 §5 + // ActivityPointer — `record_id` resolving through the object the sibling + // `object_name` column names). Unlike a lookup's static target, the + // target object is a per-row fact, so each authored row derives its own + // reference entry and then flows through the SAME resolution loop below + // — same insertedRecords/DB probes, same invalid/deferred/unresolved + // branches, same counters — one contract for every reference kind. + const pointerPairs = this.pointerRefsByObject.get(objectName) ?? []; + const derivedPointerRefs: Array<{ field: string; targetObject: string; targetField: string }> = []; + for (const pair of pointerPairs) { + const pointerValue = record[pair.field]; + // Empty pointer = no pointer: nothing to resolve, nothing to refuse. + if (pointerValue === undefined || pointerValue === null || pointerValue === '') continue; + const targetName = record[pair.objectField]; + if (typeof targetName !== 'string' || targetName === '') { + // UN-ADDRESSABLE: the id half is authored but the type half names no + // object, so no pass — now or later — can ever resolve it. Refuse + // the record rather than store a pointer that attaches to nothing + // (the exact silent failure #11339 measured). Same three registers + // as the final unresolvable branch below: counted, reported, logged. + const error: ReferenceResolutionError = { + sourceObject: objectName, + field: pair.field, + targetObject: `(missing ${pair.objectField})`, + targetField: '(pointer pair)', + attemptedValue: pointerValue, + recordIndex: i, + message: + `Pointer ${objectName}.${pair.field} = '${String(pointerValue)}' cannot be addressed: its pair ` + + `field \`${pair.objectField}\` is empty on record #${i}, so there is no object to resolve the id ` + + `against. Author \`${pair.objectField}\` with the target object's machine name, or drop the ` + + `\`${pair.field}\` value.`, + }; + errors.push(error); + allErrors.push(error); + // Dry-run stays QUIET beyond the report, like the unresolved branch + // below (#4997): nothing was written, the caller reads the result. + if (!config.dryRun) { + this.logger.error( + `[SeedLoader] ${error.message} ${objectName} record #${i} was NOT seeded — a row whose pointer ` + + `names no object would render on no timeline and match no console filter.`, + undefined, + { object: objectName, field: pair.field, recordIndex: i }, + ); + unresolvedRefError = true; + } + continue; + } + // The pair's targetField follows the same rule buildReferenceMap + // applies to static references: the target DATASET's declared + // externalId when this load carries one (single-field only), else the + // historical 'name' default — resolved here because only the row + // knows which object it points at. + const declaredExternalId = this.seedExternalIdByObject.get(targetName); + derivedPointerRefs.push({ + field: pair.field, + targetObject: targetName, + targetField: typeof declaredExternalId === 'string' ? declaredExternalId : DEFAULT_EXTERNAL_ID_FIELD, + }); + } + + const recordRefs: Array< + Pick & { multiple?: boolean } + > = derivedPointerRefs.length === 0 ? objectRefs : [...objectRefs, ...derivedPointerRefs]; + for (const ref of recordRefs) { // Never re-resolve the tenant stamp we just wrote (see above). A seed // that authors `organization_id` ITSELF still goes through resolution, // so naming an org by its natural key keeps working. diff --git a/packages/plugins/plugin-audit/src/objects/sys-activity.object.ts b/packages/plugins/plugin-audit/src/objects/sys-activity.object.ts index 01e2e8dce7..f3e27bb4bd 100644 --- a/packages/plugins/plugin-audit/src/objects/sys-activity.object.ts +++ b/packages/plugins/plugin-audit/src/objects/sys-activity.object.ts @@ -126,6 +126,12 @@ export const SysActivity = ObjectSchema.create({ required: false, readonly: true, searchable: true, + // [#11339] The id half of the ActivityPointer pair (ADR-0052 §5): a + // record id of the object `object_name` names on the same row. Declaring + // it makes the pair seedable — a packaged app's seed writes the target's + // natural key and the loader resolves it through `object_name`, instead + // of storing a literal that attaches to nothing. + referenceVia: 'object_name', group: 'Target', }), @@ -165,6 +171,9 @@ export const SysActivity = ObjectSchema.create({ searchable: true, maxLength: 255, description: 'Record id of the rich source entity (paired with source_object) — lets the timeline drill to the full email/call/meeting record.', + // [#11339] Second ActivityPointer pair — same seed-time resolution + // through the sibling `source_object` column. + referenceVia: 'source_object', group: 'Target', }), diff --git a/packages/spec/authorable-surface/data.json b/packages/spec/authorable-surface/data.json index 4da4d5b4b8..f2d1806ef7 100644 --- a/packages/spec/authorable-surface/data.json +++ b/packages/spec/authorable-surface/data.json @@ -387,6 +387,7 @@ "data/Field:readonly", "data/Field:readonlyWhen", "data/Field:reference", + "data/Field:referenceVia", "data/Field:relatedList", "data/Field:relatedListColumns", "data/Field:relatedListFilter", diff --git a/packages/spec/liveness/field.json b/packages/spec/liveness/field.json index 2fc020ea4b..d7fb0de375 100644 --- a/packages/spec/liveness/field.json +++ b/packages/spec/liveness/field.json @@ -194,6 +194,12 @@ "evidence": "packages/objectql/src/engine.ts:1672", "note": "CAVEAT — $expand/cascade/seed live; FK DDL reads reference_to (unmapped from reference)." }, + "referenceVia": { + "status": "live", + "verifiedAt": "2026-08-23", + "evidence": "packages/metadata-protocol/src/seed-loader.ts (load step 4.5 collects declared pointer pairs; loadDataset derives a per-row reference through the sibling column and routes it through the shared resolution loop — same probes, pass-2 deferral, loud refusal); packages/plugins/plugin-audit/src/objects/sys-activity.object.ts (record_id via object_name, source_id via source_object)", + "note": "[#11339] Polymorphic pointer pair (ADR-0052 §5 ActivityPointer): marks a text field as the id half whose target object the SIBLING column names per row. Consumer (seed-time natural-key resolution + loud refusal of unresolvable/un-addressable pointers) lands in the same PR (ADR-0049 declare = enforce; the maskingRule precedent). Authoring contradictions refused at parse (text-only, exclusive with reference) and create (sibling must be declared). Proven in packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts and packages/spec/src/data/field.test.ts." + }, "autonumberFormat": { "status": "live", "evidence": "packages/objectql/src/engine.ts:765", diff --git a/packages/spec/liveness/state-counts.md b/packages/spec/liveness/state-counts.md index f34872847d..a8bd0c475f 100644 --- a/packages/spec/liveness/state-counts.md +++ b/packages/spec/liveness/state-counts.md @@ -28,7 +28,7 @@ for both corollaries. | Type | live | exp | dead | planned | classified | |---|---|---|---|---|---| | `object` | 50 | 0 | 0 | 1 | 51 | -| `field` | 88 | 0 | 0 | 2 | 90 | +| `field` | 89 | 0 | 0 | 2 | 91 | | `flow` | 34 | 0 | 6 | 0 | 40 | | `action` | 42 | 0 | 2 | 2 | 46 | | `hook` | 18 | 0 | 2 | 0 | 20 | @@ -58,4 +58,4 @@ for both corollaries. | `capability` | 12 | 0 | 0 | 0 | 12 | | `qa` | 4 | 0 | 5 | 0 | 9 | | `manifest` | 22 | 0 | 21 | 0 | 43 | -| **total** | **820** | **5** | **76** | **11** | **912** | +| **total** | **821** | **5** | **76** | **11** | **913** | diff --git a/packages/spec/src/data/field.test.ts b/packages/spec/src/data/field.test.ts index 8d6fb09793..38a153cfe2 100644 --- a/packages/spec/src/data/field.test.ts +++ b/packages/spec/src/data/field.test.ts @@ -1552,3 +1552,100 @@ describe('FieldSchema — `maskingRule` is a DECLARED key (#8993, ruled Option A expect(prop!.description).toMatch(/FieldMasker/); }); }); + +describe('Polymorphic pointer pair — referenceVia (#11339, ADR-0052 §5)', () => { + it('accepts referenceVia on a text field (the ActivityPointer id half)', () => { + const field: Field = { + name: 'record_id', + label: 'Record ID', + type: 'text', + referenceVia: 'object_name', + }; + const result = FieldSchema.parse(field); + expect(result.referenceVia).toBe('object_name'); + }); + + it('rejects referenceVia on a non-text field, naming the fix', () => { + const result = FieldSchema.safeParse({ + name: 'record_id', + label: 'Record ID', + type: 'lookup', + referenceVia: 'object_name', + }); + expect(result.success).toBe(false); + const issue = result.error!.issues.find((i) => i.path.join('.') === 'referenceVia'); + expect(issue).toBeDefined(); + expect(issue!.message).toMatch(/text/); + expect(issue!.message).toMatch(/lookup/); + }); + + it('rejects referenceVia combined with reference — a static and a per-row target contradict', () => { + const result = FieldSchema.safeParse({ + name: 'record_id', + label: 'Record ID', + type: 'text', + reference: 'crm_lead', + referenceVia: 'object_name', + }); + expect(result.success).toBe(false); + const issue = result.error!.issues.find((i) => i.path.join('.') === 'referenceVia'); + expect(issue).toBeDefined(); + expect(issue!.message).toMatch(/reference/); + }); + + it('rejects a non-machine-name sibling spelling', () => { + const result = FieldSchema.safeParse({ + name: 'record_id', + label: 'Record ID', + type: 'text', + referenceVia: 'Object Name', + }); + expect(result.success).toBe(false); + }); + + it('ObjectSchema.create refuses a referenceVia whose sibling is not declared, listing the declared fields', () => { + expect(() => + ObjectSchema.create({ + name: 'sys_pointer_test', + fields: { + summary: { type: 'text', label: 'Summary' }, + record_id: { type: 'text', label: 'Record ID', referenceVia: 'object_name' }, + }, + }), + ).toThrow(/no field named "object_name"/); + }); + + it('ObjectSchema.create refuses a referenceVia pointing at itself', () => { + expect(() => + ObjectSchema.create({ + name: 'sys_pointer_test', + fields: { + record_id: { type: 'text', label: 'Record ID', referenceVia: 'record_id' }, + }, + }), + ).toThrow(/ITSELF/); + }); + + it('ObjectSchema.create accepts the canonical declared pair', () => { + const obj = ObjectSchema.create({ + name: 'sys_pointer_test', + fields: { + object_name: { type: 'text', label: 'Object' }, + record_id: { type: 'text', label: 'Record ID', referenceVia: 'object_name' }, + }, + }); + expect((obj.fields as Record).record_id.referenceVia).toBe('object_name'); + }); + + it('the `.describe()` prose states the seed-time contract and the non-goals (feeds the reference page)', () => { + const js = z.toJSONSchema(FieldSchema as unknown as z.ZodType, { + unrepresentable: 'any', + io: 'input', + }) as { properties?: Record }; + const prop = js.properties?.referenceVia; + expect(prop).toBeDefined(); + expect(prop!.description).toMatch(/sibling/i); + expect(prop!.description).toMatch(/seed/i); + expect(prop!.description).toMatch(/referential integrity/); + }); +}); diff --git a/packages/spec/src/data/field.zod.ts b/packages/spec/src/data/field.zod.ts index 49b46f0b0c..1757b6a224 100644 --- a/packages/spec/src/data/field.zod.ts +++ b/packages/spec/src/data/field.zod.ts @@ -925,6 +925,33 @@ export const FieldSchema = lazySchema(() => strictObject({ 'Target object name (snake_case) for lookup/master_detail fields. ' + 'Required for relationship types. Used by $expand to resolve foreign key IDs into full objects.' ), + /** + * Polymorphic pointer declaration (ADR-0052 §5 — the ActivityPointer model). + * + * Marks this `text` field as the ID HALF of a two-column pointer pair: its + * value is a record id of the object NAMED BY the sibling field this key + * points at — `record_id: Field.text({ referenceVia: 'object_name' })` on + * `sys_activity` is the canonical pair. The target object varies per ROW + * (read from the sibling column), which is exactly what the static + * `reference` key cannot express; the two are mutually exclusive. + * + * What declaring it ENFORCES today (#11339): seed loading resolves the + * authored value as a natural key against the object the sibling column + * names — the same externalId machinery lookup references use — and treats + * an unresolvable pointer as the loud reference failure it is, instead of + * storing the literal string as a row that attaches to nothing. It does NOT + * (yet) add referential integrity, cascade behavior, or $expand support — + * consumers that do not read this key keep treating the field as plain text. + */ + referenceVia: z.string().regex(/^[a-z_][a-z0-9_]*$/).optional().describe( + 'Declares this text field as the id half of a polymorphic pointer pair (ADR-0052 §5 ActivityPointer): ' + + 'the value is a record id of the object named by the SIBLING FIELD this key names — e.g. ' + + "`record_id` with `referenceVia: 'object_name'`. The sibling must be a declared field on the same " + + 'object holding an object machine name. Text fields only; mutually exclusive with `reference` (a ' + + 'static and a per-record target contradict). Enforced today at seed load: the value resolves as a ' + + 'natural key against the object the sibling column names, and an unresolvable pointer is refused ' + + 'loudly instead of stored verbatim. Adds no referential integrity or $expand behavior.' + ), // `referenceFilters` (string[]) removed in the 16.x line (#2377, ADR-0049): // the lookup picker reads the structured `lookupFilters` ({field,operator,value}), // never this string[] form — as authored it filtered nothing. Use `lookupFilters`. @@ -1438,6 +1465,34 @@ export const FieldSchema = lazySchema(() => strictObject({ // return) — so it has been a known gap longer than any of its siblings. ...MetadataProtectionFields, }).superRefine((field, ctx) => { + // [#11339] `referenceVia` declares the id half of a polymorphic pointer + // pair (ADR-0052 §5) — semantics only a plain `text` column carries. On a + // relationship type it contradicts the type's own single static target, and + // on any other type the stored value could not hold a record id. Refused at + // the authoring seam, where the fix is one keystroke away. + if (field.referenceVia !== undefined && field.type !== 'text') { + ctx.addIssue({ + code: 'custom', + path: ['referenceVia'], + message: + `\`referenceVia\` is only valid on \`type: 'text'\` (this field is \`${field.type}\`): it marks a ` + + 'plain text column as the id half of a polymorphic pointer pair whose target object is named per ' + + "row by a sibling column (e.g. `record_id` with `referenceVia: 'object_name'`). For a fixed " + + 'target use a `lookup`/`master_detail` field with `reference` instead.', + }); + } + if (field.referenceVia !== undefined && field.reference !== undefined) { + ctx.addIssue({ + code: 'custom', + path: ['referenceVia'], + message: + '`referenceVia` cannot be combined with `reference`: one declares a per-row target read from a ' + + 'sibling column, the other a single static target object — both cannot be honest about where ' + + 'this field points. Keep `referenceVia` for a polymorphic pointer, or `reference` (on a ' + + 'lookup/master_detail field) for a fixed relationship.', + }); + } + // ADR-0113: `storage.notNull` × `requiredWhen` is a contradiction, rejected // at the authoring seam — when the condition is FALSE the write contract // permits null, but the column would refuse it, so the author has declared diff --git a/packages/spec/src/data/object.zod.ts b/packages/spec/src/data/object.zod.ts index 555c640285..72a71d8e4a 100644 --- a/packages/spec/src/data/object.zod.ts +++ b/packages/spec/src/data/object.zod.ts @@ -2419,6 +2419,48 @@ function assertSystemDataIsWritable( ); } +/** + * [#11339] A field's `referenceVia` must name a DECLARED sibling field — + * the type half of a polymorphic pointer pair (ADR-0052 §5; `record_id` with + * `referenceVia: 'object_name'` on `sys_activity` is the canonical pair). + * + * Why here: `FieldSchema` sees one field at a time, so "the sibling exists" + * is only checkable at the object level. A pointer whose declared sibling is + * missing can never resolve — every seed row would fail at load time with a + * per-row error, far from the one-line declaration mistake that caused it. + * Refusing at `create()` puts the error where the fix is. + * + * Lives at `create()` — the authoring surface (ADR-0077) — beside + * {@link assertSystemDataIsWritable}, and deliberately NOT in raw + * `.parse()`/`.safeParse()`: metadata already at rest must keep loading; the + * seed loader independently reports an un-addressable pointer per row. + */ +function assertReferenceViaSiblingDeclared(objectName: unknown, fields: unknown): void { + if (fields === null || typeof fields !== 'object') return; + const name = typeof objectName === 'string' && objectName.length > 0 ? objectName : ''; + const fieldMap = fields as Record; + for (const [fieldName, fieldDef] of Object.entries(fieldMap)) { + const via = fieldDef?.referenceVia; + if (typeof via !== 'string') continue; + if (via === fieldName) { + throw new Error( + `ObjectSchema.create('${name}'): field "${fieldName}" declares \`referenceVia: '${via}'\` — ` + + 'pointing at ITSELF. `referenceVia` names the sibling field that holds the target OBJECT ' + + "machine name (the type half of the pointer pair), e.g. `record_id` with `referenceVia: " + + "'object_name'`; a field cannot be both halves.", + ); + } + if (!Object.prototype.hasOwnProperty.call(fieldMap, via)) { + throw new Error( + `ObjectSchema.create('${name}'): field "${fieldName}" declares \`referenceVia: '${via}'\`, but ` + + `this object declares no field named "${via}". \`referenceVia\` names the sibling field that ` + + 'holds the target object machine name per row (ADR-0052 §5 pointer pair) — declare that ' + + `sibling, or fix the spelling. Declared fields: ${Object.keys(fieldMap).join(', ')}.`, + ); + } + } +} + /** * [#9138 — #8772 maintainer ruling, Direction 2 / ADR-0055] Under * `sharingModel: 'controlled_by_parent'` the builder FORCES `required: true` @@ -2608,6 +2650,10 @@ export const ObjectSchema = lazySchema(() => { // contradiction with no honest reading — refuse it here, where it is cheap // to fix, rather than shipping a bucket whose name lies again. assertSystemDataIsWritable(cfg.name, cfg.managedBy, cfg.userActions); + // [#11339] A `referenceVia` pointer pair whose type-half sibling is not + // declared can never resolve — refuse at the authoring seam, beside its + // sibling assertions, rather than one error per seeded row at load time. + assertReferenceViaSiblingDeclared(cfg.name, cfg.fields); // [#9138 — #8772 ruling, Direction 2] A `controlled_by_parent` object's // `master_detail` reference is forced `required: true` (an explicit // `required: false` throws, loudly) so the unsafe shape cannot be newly From b9af5410ab3bf99caa786a1f772b5851d996c3c9 Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Sun, 23 Aug 2026 23:11:10 +0800 Subject: [PATCH 2/2] test(seed): NodeNext-clean import + pin the new fake engine in the double-contract ledger The pointer-pair test's relative import gains its .js extension so the package's tsc debt count stays at its recorded 63 (the ratchet refuses +1), and check:engine-double-contract --write records the new pinned coverage. Co-Authored-By: Claude Opus 5 --- .../src/seed-loader-pointer-pair.test.ts | 2 +- scripts/engine-double-contract.pinned.json | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts b/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts index 0788722839..6125a7b970 100644 --- a/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts +++ b/packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts @@ -1,7 +1,7 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. import { describe, it, expect, vi } from 'vitest'; -import { SeedLoaderService } from './seed-loader'; +import { SeedLoaderService } from './seed-loader.js'; import type { IDataEngine, IMetadataService } from '@objectstack/spec/contracts'; import { assertEngineDeleteDispatch, assertEngineUpdateDispatch } from '@objectstack/metadata-core'; diff --git a/scripts/engine-double-contract.pinned.json b/scripts/engine-double-contract.pinned.json index abec102555..e92c29611a 100644 --- a/scripts/engine-double-contract.pinned.json +++ b/scripts/engine-double-contract.pinned.json @@ -806,6 +806,16 @@ "verb": "update", "pinned": 1 }, + { + "file": "packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts", + "verb": "delete", + "pinned": 1 + }, + { + "file": "packages/metadata-protocol/src/seed-loader-pointer-pair.test.ts", + "verb": "update", + "pinned": 1 + }, { "file": "packages/metadata-protocol/src/seed-loader-replay.test.ts", "verb": "delete",