Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .changeset/mapping-lookup-params-removed.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
---
"@objectstack/spec": minor
---

feat(spec): retire the import mapping `lookup` transform's steering params (#10329, ADR-0049)

<!-- adr-0087: registered mapping-lookup-params-removed -->

**BREAKING** accept-set narrowing, landing after the v17.0.0 cut (the lockstep
launch-window convention ships it as `minor`; the migration prescription is
registered under protocol major 18, where `os migrate meta` users will look).

`ImportFieldMappingSchema.params` declared four keys whose only stated purpose
was to steer the `lookup` transform — `object` ("Lookup Object"), `fromField`
("Match on"), `toField` ("Value to take"), `autoCreate` ("Create if missing") —
and the import path read none of them: `applyMappingToRows` handles `lookup` in
the same branch as `none` (the cell is copied through unchanged), and reference
resolution runs afterwards in `import-coerce.ts`, driven by the target field's
own metadata. Implementing them would be a second reference-resolution dialect
on the import path, which the code declines to build and the #10329 triage
ruling declined again. `autoCreate` was the one with teeth: it read as
"create the referenced record when nothing matches", and nothing was ever
created — with or without the key, an unresolved cell fails its row with
`import_reference_not_found`.

FROM → TO, per key (all four are pure deletions — none ever had an effect to
lose, so removing them changes no import behaviour):

- `fieldMapping[].params.object` → *(removed)*. The referenced object is the
target field's own declared `reference`.
- `fieldMapping[].params.fromField` → *(removed)*. The pipeline matches the
cell's display value against the referenced object itself.
- `fieldMapping[].params.toField` → *(removed)*. Resolution always writes the
referenced record's id.
- `fieldMapping[].params.autoCreate` → *(removed)*. Create or import the
referenced records first, then import the rows that point at them.

One-line fix: delete the four keys (and any of their old alias spellings) from
`fieldMapping[].params`; `os migrate meta --from 17` lists the mechanical edits
for existing sources.

The retirement kit:

- `params` is a `strictObject`, so the keys are strict-deleted with a
`guidance` prescription per spelling — the four canonical keys and the
eleven ex-alias spellings (`lookupObject`/`targetObject`,
`match`/`matchOn`/`matchField`/`keyField`, `returnField`/`valueField`,
`create`/`createIfMissing`/`upsert`) all land on the full prescription
rather than a "did you mean" pointing at a key that is also gone (the
17.0.0 #4509 treatment, one level down)
- ADR-0087 registration: the D2 conversion `mapping-lookup-params-removed`
(protocol 18), wired into the step-18 chain — `os migrate meta --from 17`
strips the four keys from stored `mappings[].fieldMapping[].params`. No
retired-key table entries: these keys sit one sub-walk level below the
authorable-surface drill (`data/ImportFieldMapping:params` is the walked
row, and it stays live), so there is no `defKey:name` row to register or
age out
- pin tests (`mapping.test.ts`): refusal pins per key asserting the
prescription (the `autoCreate` pin asserts the row-fails truth), alias
routing pins, and a surviving-surface pin (`value`/`valueMap`/`separator`
untouched)
- liveness ledger: `liveness/mapping.json`'s `fieldMapping` sub-walk boundary
note now records the retirement instead of parking the finding
- docs: the `import-mappings.mdx` warning about the inert params is deleted
along with the keys; the generated mapping reference no longer lists them
8 changes: 0 additions & 8 deletions content/docs/data-modeling/import-mappings.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -129,14 +129,6 @@ path, one is a deliberate pass-through, and one is refused:
| `lookup` | **Pass-through.** The cell is copied unchanged, and the import's own reference resolution turns the display text into a record id afterwards — see [After the mapping](#after-the-mapping-cell-coercion). | — |
| `javascript` | **Refused.** There is no server-side sandbox, and silently skipping a declared transform would corrupt data. `defineStack()` fails the build; a runtime-saved mapping is rejected by the import request with `400 UNSUPPORTED_TRANSFORM`. | — |

<Callout type="warn">
The `lookup` transform's own `params` keys (`object`, `fromField`, `toField`,
`autoCreate`) parse, but the import path reads none of them: reference resolution is
driven by the **target object's field definitions**, not by the mapping. Do not write
them expecting them to steer anything — tracked as
[#10329](https://github.com/objectstack-ai/objectstack/issues/10329).
</Callout>

For logic beyond these, transform the data before you post it, or model it as a
[flow](/docs/automation) on the target object.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/mapping.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -30,7 +30,7 @@ const result = ImportFieldMappingSchema.parse(data);
| **source** | `string \| string[]` | ✅ | Source column header(s) |
| **target** | `string \| string[]` | ✅ | Target object field(s) |
| **transform** | `Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>` | optional (default: `"none"`) | |
| **params** | `{ value?: any; object?: string; fromField?: string; toField?: string; … }` | optional | |
| **params** | `{ value?: any; valueMap?: Record<string, any>; separator?: string }` | optional | |


---
Expand Down
2 changes: 1 addition & 1 deletion packages/spec/liveness/mapping.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -30,7 +30,7 @@
"status": "live",
"verifiedAt": "2026-08-01",
"evidence": "packages/rest/src/import-mapping.ts:98-105, packages/rest/src/import-mapping.ts:115-167",
"note": "the pipeline itself: source/target/transform/params all consumed. none/constant/map/split/join applied in applyMappingToRows (`params.separator` :124, `.value` :132, `.valueMap` :137); `lookup` copies through for the pipeline's metaMap reference resolution; `javascript` is REJECTED with a 400 (no server sandbox — implement-or-reject-loudly). SUB-WALK BOUNDARY, recorded not hidden: `params`' lookup-specific keys (`object`/`fromField`/`toField`/`autoCreate`) are read by nothing — reference resolution comes from the target object's own field definitions, not from these — and they sit one level below the drill, so only this note governs them."
"note": "the pipeline itself: source/target/transform/params all consumed. none/constant/map/split/join applied in applyMappingToRows (`params.separator` :124, `.value` :132, `.valueMap` :137); `lookup` copies through for the pipeline's metaMap reference resolution; `javascript` is REJECTED with a 400 (no server sandbox — implement-or-reject-loudly). SUB-WALK BOUNDARY, resolved: `params`' lookup-specific keys (`object`/`fromField`/`toField`/`autoCreate`) were read by nothing — reference resolution comes from the target object's own field definitions, not from these — and were REMOVED in the 17.x line (#10329, ADR-0049 enforce-or-remove; strict deletion, so nothing dead remains one level below the drill). Every spelling — the four canonical keys and their eleven ex-aliases — now lands on a guidance prescription at parse; the mapping-lookup-params-removed conversion (protocol 18) strips them from stored sources. The surviving params keys (`value`/`valueMap`/`separator`) are all consumed by applyMappingToRows."
},
"mode": {
"status": "live",
Expand Down
115 changes: 115 additions & 0 deletions packages/spec/src/conversions/registry.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -7556,6 +7556,120 @@ const recordHighlightsFieldIconRemoved: MetadataConversion = {
},
};

/**
* `mapping.fieldMapping[].params` lookup keys removed (#10329, ADR-0049
* enforce-or-remove — the sub-walk half of the 17.0.0 #4509 mapping cleanup).
*
* `object` / `fromField` / `toField` / `autoCreate` declared a per-entry
* reference-resolution dialect that the import path never implemented:
* `applyMappingToRows` handles `lookup` in the same branch as `none` (the cell
* is copied through unchanged), and reference resolution runs afterwards in
* `import-coerce.ts` off the TARGET FIELD's own metadata — never off these
* keys. Implementing them (a second reference-resolution dialect on the import
* path) is what the code comment in `packages/rest/src/import-mapping.ts`
* declines to build, and the #10329 triage ruling confirms that posture.
*
* `autoCreate` was the one with teeth: it read as "create the referenced
* record when nothing matches", and nothing was ever created — with or without
* the key, an unresolved cell fails its row with `import_reference_not_found`.
* The schema prescription says so outright, because an author who believed the
* key is one support ticket away from "my import used to create these".
*
* **A pure lossless delete.** None of the four ever had an effect to lose, so
* stripping them preserves observed import behaviour exactly.
*
* Scoped to the `mappings` collection, then drilled two levels down
* (`fieldMapping[]` is an ARRAY one level below the item, `params` a dict one
* below that), so the top-level-only `stripKeys` runs per entry's params bag —
* the `metric-filters-removed` shape, one level deeper. Deliberately narrow:
* `object` / `targetObject`-style keys are live all over the tree, and a
* stack-wide strip would delete enforced keys from other types.
*/
const mappingLookupParamsRemoved: MetadataConversion = {
id: 'mapping-lookup-params-removed',
toMajor: 18,
retiredFromLoadPath: true,
surface: 'mapping.fieldMapping[].params.object / .fromField / .toField / .autoCreate',
summary:
"mapping lookup params 'object'/'fromField'/'toField'/'autoCreate' removed (#10329, "
+ 'ADR-0049 — the import path never read them: `lookup` copies the cell through and '
+ "reference resolution runs off the target field's own metadata. `autoCreate` never "
+ 'created anything — an unresolved reference fails the row either way)',
apply(stack, emit) {
const RETIRED = ['object', 'fromField', 'toField', 'autoCreate'];
return mapCollection(stack, 'mappings', (m, path) => {
const entries = m.fieldMapping;
if (!Array.isArray(entries)) return m;
let touched = false;
const nextEntries = entries.map((entry, i) => {
if (!isDict(entry)) return entry;
const params = entry.params;
if (!isDict(params)) return entry;
const stripped = stripKeys(params, RETIRED, emit, `${path}.fieldMapping[${i}].params`);
if (stripped === params) return entry;
touched = true;
return { ...entry, params: stripped };
});
if (!touched) return m;
return { ...m, fieldMapping: nextEntries };
});
},
fixture: {
before: {
mappings: [{
name: 'csv_import_contacts',
targetObject: 'contact',
fieldMapping: [
// The retired shape: a lookup entry steering nothing.
{
source: 'Account Name',
target: 'account_id',
transform: 'lookup',
params: { object: 'account', fromField: 'name', toField: 'id', autoCreate: true },
},
// A live params bag on another transform rides through untouched —
// the strip dispatches on key presence, and copy-on-write keeps the
// reference.
{
source: 'Status',
target: 'status',
transform: 'map',
params: { valueMap: { Open: 'open' } },
},
// No params at all — untouched.
{ source: 'Email', target: 'email' },
],
}],
},
after: {
mappings: [{
name: 'csv_import_contacts',
targetObject: 'contact',
fieldMapping: [
// The emptied bag stays: the conversion strips KEYS, and deleting
// the bag itself would be a second, unprescribed edit. `params: {}`
// parses clean.
{
source: 'Account Name',
target: 'account_id',
transform: 'lookup',
params: {},
},
{
source: 'Status',
target: 'status',
transform: 'map',
params: { valueMap: { Open: 'open' } },
},
{ source: 'Email', target: 'email' },
],
}],
},
// Four notices: one per retired key on the single lookup entry.
expectedNotices: 4,
},
};

export const CONVERSIONS_BY_MAJOR: Readonly<Record<number, readonly MetadataConversion[]>> = {
11: [flowNodeHttpRename, pageKindJsxToHtml, flowNodeFilterAlias, objectCompactLayoutRename],
13: [stackRolesToPositions, owdLegacyReadAliases, sharingRecipientRoleToPosition],
Expand DownExpand Up@@ -7637,6 +7751,7 @@ export const CONVERSIONS_BY_MAJOR: Readonly<Record<number, readonly MetadataConv
fieldColumnListsCanonicalized,
metricFiltersRemoved,
recordHighlightsFieldIconRemoved,
mappingLookupParamsRemoved,
],
};

Expand Down
98 changes: 82 additions & 16 deletions packages/spec/src/data/mapping.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -87,23 +87,92 @@ describe('ImportFieldMappingSchema', () => {
expect(mapping.params?.value).toBe('active');
});

it('should accept lookup transform', () => {
it('should accept lookup transform (a pass-through with no params of its own)', () => {
const mapping = ImportFieldMappingSchema.parse({
source: 'account_name',
target: 'account_id',
transform: 'lookup',
params: {
object: 'account',
fromField: 'name',
toField: 'id',
autoCreate: false
}
});

expect(mapping.transform).toBe('lookup');
expect(mapping.params?.object).toBe('account');
expect(mapping.params?.fromField).toBe('name');
expect(mapping.params?.toField).toBe('id');
});

// ── `params` lookup keys retired in the 17.x line (#10329, ADR-0049) ───────
//
// `object` / `fromField` / `toField` / `autoCreate` declared a per-entry
// reference-resolution dialect the import path never implemented: `lookup`
// copies the cell through and resolution runs afterwards off the target
// field's own metadata. These pin the REJECTION carrying the prescription,
// not just the absence — the schema is strict, so a bare "unrecognized key"
// would already fail; what must survive refactors is the guidance.

const lookupEntry = { source: 'account_name', target: 'account_id', transform: 'lookup' as const };

it('rejects the retired `params.object` with the target-field-metadata prescription', () => {
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry,
params: { object: 'account' },
})).toThrow(/params\.object.*removed.*TARGET FIELD/s);
});

it('rejects the retired `params.fromField` with the display-value-matching prescription', () => {
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry,
params: { fromField: 'name' },
})).toThrow(/params\.fromField.*removed.*display.*value/s);
});

it('rejects the retired `params.toField` with the record-id prescription', () => {
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry,
params: { toField: 'id' },
})).toThrow(/params\.toField.*removed.*record.*id/s);
});

it('rejects the retired `params.autoCreate` saying what ACTUALLY happens (row fails)', () => {
// The one with teeth: the key read as "create the referenced record when
// nothing matches". Nothing was ever created — the row fails with an
// unresolved-reference error either way — so the prescription must say so
// rather than merely "removed".
const parse = () => ImportFieldMappingSchema.parse({
...lookupEntry,
params: { autoCreate: true },
});
expect(parse).toThrow(/params\.autoCreate.*removed/s);
expect(parse).toThrow(/nothing was ever created/is);
expect(parse).toThrow(/import_reference_not_found/s);
});

it('routes the retired ALIAS spellings to the same prescriptions', () => {
// The alias table used to fold eleven spellings onto the four keys. Leaving
// them as aliases would answer "did you mean `fromField`?" — a rename
// suggestion pointing at a key that is also gone, i.e. a second rejection.
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry, params: { lookupObject: 'account' },
})).toThrow(/params\.object.*removed/s);
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry, params: { matchOn: 'name' },
})).toThrow(/params\.fromField.*removed/s);
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry, params: { returnField: 'id' },
})).toThrow(/params\.toField.*removed/s);
expect(() => ImportFieldMappingSchema.parse({
...lookupEntry, params: { createIfMissing: true },
})).toThrow(/params\.autoCreate.*removed/s);
});

it('leaves the surviving params surface intact', () => {
const mapping = ImportFieldMappingSchema.parse({
source: 'status',
target: 'status_code',
transform: 'map',
params: { valueMap: { Open: 'open' } },
});
expect(mapping.params).not.toHaveProperty('object');
expect(mapping.params).not.toHaveProperty('fromField');
expect(mapping.params).not.toHaveProperty('toField');
expect(mapping.params).not.toHaveProperty('autoCreate');
expect(mapping.params?.valueMap).toHaveProperty('Open', 'open');
});

it('should accept map transform', () => {
Expand DownExpand Up@@ -375,14 +444,11 @@ describe('MappingSchema', () => {
params: { value: 'active' }
},
{
// `lookup` is a pass-through: the import pipeline resolves the
// reference from the target field's own metadata (#10329).
source: 'account_name',
target: 'account_id',
transform: 'lookup',
params: {
object: 'account',
fromField: 'name',
toField: 'id'
}
transform: 'lookup'
},
{
source: ['first_name', 'last_name'],
Expand Down
Loading
Loading