Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .changeset/filesystem-loader-keyed-items.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
---
'@objectstack/metadata': patch
---

`FilesystemLoader` now implements `loadManyKeyed()`, so a metadata file whose
body has no top-level `name` is no longer invisible to `MetadataManager.list()`.
`loadMany()` globbed files and pushed bodies, discarding the path it had just
read; the manager then fell back to keying by `body.name`, which drops every
nameless body — an aggregated `defineView` container has none by design. That is
the #14205 defect, unrepaired for this loader until now.

The key is this loader's own name-to-path derivation — the basename minus
extension, the same one `list()` reports — but only where that derivation is a
bijection for the file: it sits directly under `ROOT/TYPE/` and carries an
extension `findFile()` tries, so `findFile(type, key)` resolves back to that same
file. Every other shape (a nested path, an extension-less file) keeps the
previous behaviour verbatim: keyed by `body.name` when it has one, dropped when
it has none. `list()` and `findFile()` disagree outside the flat shape — `list()`
reports the bare basename for a nested file and `findFile()` cannot resolve it —
so keying those by the basename would mint names `get()` and `exists()` cannot
open, and two directories holding one basename would collide silently. Repairing
the derivation itself is tracked separately.

One deliberate consequence: a flat file whose `body.name` disagrees with its
basename is now keyed by the basename. That is #14205's rule (identity is the
key the store holds an item under, not `body.name`) applied to this loader, and
it aligns `list()` with `listNames()` for that shape. `loadMany()`'s own
signature and answer are unchanged; both methods now share one file walk so
their bodies cannot drift.
240 changes: 240 additions & 0 deletions packages/metadata/src/loaders/filesystem-loader-keyed-items.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #14341 — `FilesystemLoader.loadManyKeyed()`: a file-held item is keyed by the
* name this loader can actually RESOLVE for it, and by nothing else.
*
* ---------------------------------------------------------------------------
* The defect
* ---------------------------------------------------------------------------
* `loadMany()` globbed files and pushed bodies, throwing away the path it had
* just read. `MetadataManager.admitLoaderItems()` then fell back to keying by
* `body.name`, which drops every body that has no top-level `name` — the exact
* #14205 failure, unrepaired for this loader. A `defineView` container has no
* own `name` BY DESIGN, so a file holding one was absent from `list('view')`
* while `listDiagnosed()` called the short answer complete.
*
* ---------------------------------------------------------------------------
* The rule this pins (PM ruling on #14341, 2026-09-02 — option D)
* ---------------------------------------------------------------------------
* An item is keyed by this loader's own name-to-path derivation (the basename
* minus extension, the same derivation `list()` reports) ONLY where that
* derivation is a BIJECTION for the file — it sits directly under `ROOT/TYPE/`
* and carries an extension `findFile()` tries, so `findFile(type, key)` resolves
* back to that same file. Every other shape keeps the pre-#14205 behaviour
* verbatim: keyed by `body.name` when it has one, dropped when it has none.
*
* The ruling was taken over triage's "a nested path keeps whatever `list()`
* reports for it today", knowingly, because the two derivations DISAGREE
* outside the flat shape (measured on `origin/main` @ 253da34c4): `list()`
* reports `account` for `ROOT/TYPE/crm/account.json`, and `findFile()` resolves
* that name against `ROOT/TYPE/account.json` and finds nothing. Keying nested
* files by their basename would mint names `get()` / `load()` / `exists()`
* cannot open, and two directories holding one basename would collide in
* silence. The card's own fence: "keying items under names nothing else uses
* ... is worse than today's honest drop".
*
* ---------------------------------------------------------------------------
* What the RECORD cases are for
* ---------------------------------------------------------------------------
* `RECORD:` cases pin behaviour this ruling deliberately LEAVES ALONE — the
* nested nameless file is still dropped. They exist so the derivation repair
* (#14486: one shared name-to-path function for `list()`, `findFile()` and
* `loadManyKeyed()`) inverts them deliberately, with the change visible in a
* diff, instead of silently.
*
* `CONTROL:` cases are green in both directions and pin "nothing consumers see
* today changes shape".
*/

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import type { MetadataFormat } from '@objectstack/spec/system';
import { MetadataManager } from '../metadata-manager.js';
import { FilesystemLoader } from './filesystem-loader.js';
import { JSONSerializer } from '../serializers/json-serializer.js';
import type { MetadataSerializer } from '../serializers/serializer-interface.js';

const TYPE = 'view';

/** The aggregated container shape: identity is the target object, no own `name`. */
const NAMELESS_CONTAINER = { object: 'account', views: [{ label: 'All' }] };

let root: string;

/**
* One tree holding every shape the rule distinguishes. The two `crm/` files and
* the extension-less one are the shapes where `list()` and `findFile()` disagree.
*/
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'fsloader-keyed-'));
const typeDir = path.join(root, TYPE);
await fs.mkdir(path.join(typeDir, 'crm'), { recursive: true });

const write = (rel: string, body: unknown) =>
fs.writeFile(path.join(typeDir, rel), JSON.stringify(body), 'utf-8');

await write('flat_nameless.json', NAMELESS_CONTAINER);
await write('flat_named.json', { name: 'flat_named', label: 'agrees with its basename' });
await write('flat_disagreeing.json', { name: 'not_the_basename', label: 'disagrees' });
await write('dotted.config.json', { name: 'dotted.config' });
await write(path.join('crm', 'nested_named.json'), { name: 'nested_named' });
await write(path.join('crm', 'nested_nameless.json'), { ...NAMELESS_CONTAINER, object: 'lead' });
await write('extensionless', { name: 'extensionless_named' });
});

afterAll(async () => {
await fs.rm(root, { recursive: true, force: true });
});

function loader(): FilesystemLoader {
const serializers = new Map<MetadataFormat, MetadataSerializer>([
['json', new JSONSerializer()],
]);
return new FilesystemLoader(root, serializers);
}

/** A cold manager — empty registry, one filesystem loader answering. */
function coldManager(): MetadataManager {
const manager = new MetadataManager({ formats: ['json'], loaders: [] });
manager.registerLoader(loader());
return manager;
}

async function keys(): Promise<string[]> {
const keyed = await loader().loadManyKeyed(TYPE);
return keyed.map(entry => entry.name).sort();
}

describe('#14341 FilesystemLoader.loadManyKeyed() keys by the resolvable name', () => {
it('keys a FLAT file by its basename, the derivation list() reports', async () => {
expect(await keys()).toEqual(
['dotted.config', 'extensionless_named', 'flat_disagreeing', 'flat_named', 'flat_nameless', 'nested_named'],
);
});

it('admits a flat NAMELESS body, keyed by the file name', async () => {
// Pre-repair this body had no key at all and fell out of the merge.
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless');

expect(entry).toBeDefined();
expect(entry!.data).toEqual(NAMELESS_CONTAINER);
});

it('never synthesises a name into a body that deliberately has none', async () => {
const keyed = await loader().loadManyKeyed(TYPE);
const entry = keyed.find(item => item.name === 'flat_nameless')!;

// The key travels BESIDE the body; the body stays byte-identical to disk,
// so `assertMetadataRegisterContract`'s `data.name` check keeps its meaning.
expect(Object.prototype.hasOwnProperty.call(entry.data as object, 'name')).toBe(false);
});

it('keys a flat file by its BASENAME even when body.name disagrees', async () => {
// #14205's rule applied to this loader: identity is the key the store holds
// the item under, not `body.name`. `flat_disagreeing.json` says
// `name: 'not_the_basename'`, and the store's key is the file's.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.map(entry => entry.name)).toContain('flat_disagreeing');
expect(keyed.map(entry => entry.name)).not.toContain('not_the_basename');
// ...and the disagreeing body is handed back unedited.
expect(keyed.find(entry => entry.name === 'flat_disagreeing')!.data).toEqual({
name: 'not_the_basename',
label: 'disagrees',
});
});

it('strips only the final extension, so dotted.config.json keys as dotted.config', async () => {
expect(await keys()).toContain('dotted.config');
});

it('EVERY key it mints resolves back to a file through findFile()', async () => {
// The bijection claim itself, and the reason the disagreeing shapes below
// are NOT keyed by their basename: a minted key that `exists()` cannot open
// is exactly what the card refused.
const fsLoader = loader();
const derived = ['dotted.config', 'flat_disagreeing', 'flat_named', 'flat_nameless'];

for (const key of derived) {
expect(await fsLoader.exists(TYPE, key)).toBe(true);
}
});

it('keys a NESTED file by body.name — the pre-#14205 behaviour, unchanged', async () => {
// `list()` reports `nested_named` for it too, but `findFile()` resolves that
// name against `ROOT/view/nested_named.json`, which does not exist: the
// derivation is not a bijection here, so it is not used.
const fsLoader = loader();

expect(await keys()).toContain('nested_named');
expect(await fsLoader.exists(TYPE, 'nested_named')).toBe(false);
expect(await fsLoader.exists(TYPE, path.join('crm', 'nested_named'))).toBe(true);
});

it('RECORD: a nested NAMELESS file is still dropped — the honest drop, #14486', async () => {
// Not a repair this ruling makes: there is no name for it that any other
// door reports. #14486 (one shared name-to-path derivation) is where this
// inverts, deliberately.
const keyed = await loader().loadManyKeyed(TYPE);

expect(keyed.some(entry => (entry.data as { object?: string }).object === 'lead')).toBe(false);
});

it('keys an EXTENSION-LESS file by body.name — findFile() cannot resolve it either', async () => {
const fsLoader = loader();

expect(await keys()).toContain('extensionless_named');
// `findFile()` always appends one of its extensions, so the bare file name
// resolves to nothing; keying by the basename would mint a dead name.
expect(await fsLoader.exists(TYPE, 'extensionless')).toBe(false);
});

it('CONTROL: loadMany() still answers with bodies only, and with every file', async () => {
// The shared walk behind both methods must not leak its envelope, and must
// not start dropping what it read: `loadMany()`'s callers are untouched.
const items = await loader().loadMany(TYPE);

expect(items).toHaveLength(7);
for (const item of items) {
expect(Object.prototype.hasOwnProperty.call(item as object, 'file')).toBe(false);
expect(Object.prototype.hasOwnProperty.call(item as object, 'data')).toBe(false);
}
expect(items).toContainEqual(NAMELESS_CONTAINER);
});
});

describe('#14341 the repair reaches MetadataManager.list()', () => {
it('a flat nameless body reaches list() end to end', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual(NAMELESS_CONTAINER);
});

it('listDiagnosed() counts it and stays complete-and-not-degraded', async () => {
const result = await coldManager().listDiagnosed(TYPE);

// Nothing threw before the repair and nothing throws after — the loader
// answered successfully both times. What changes is that the short answer
// is no longer served as a full one.
expect(result.degraded).toBe(false);
expect(result.errors).toEqual([]);
expect(result.items).toContainEqual(NAMELESS_CONTAINER);
});

it('CONTROL: a named body is still listed, and still only once', async () => {
const items = await coldManager().list(TYPE);

expect(items).toContainEqual({ name: 'flat_named', label: 'agrees with its basename' });
expect(items.filter(item => (item as { name?: string }).name === 'flat_named')).toHaveLength(1);
});

it('RECORD: the nested nameless body is still absent from list()', async () => {
const items = await coldManager().list(TYPE);

expect(items.some(item => (item as { object?: string }).object === 'lead')).toBe(false);
});
});
Loading
Loading