From 3fdc043a89c9a6a1d748fd81da765209f796c9af Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 1 Aug 2026 07:48:05 +0000 Subject: [PATCH 1/2] =?UTF-8?q?refactor(spec)!:=20remove=20the=20kernel=20?= =?UTF-8?q?metadata-loader=20envelope=20family=20=E2=80=94=2011=20names=20?= =?UTF-8?q?declared=20twice=20with=20different=20shapes=20(#4411)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `@objectstack/spec` exported eleven names TWICE, with a different shape each time, on two subpath entries — so which type a consumer got depended on nothing but the import path: import type { MetadataWatchEvent } from '@objectstack/spec/kernel'; // one shape import type { MetadataWatchEvent } from '@objectstack/spec/system'; // another `MetadataFormat`, `MetadataStats`, `MetadataLoadOptions`, `MetadataSaveOptions`, `MetadataExportOptions`, `MetadataImportOptions`, `MetadataLoadResult`, `MetadataSaveResult`, `MetadataWatchEvent`, `MetadataCollectionInfo` and `MetadataLoaderContract` are removed from `kernel/metadata-loader.zod`. The `system/metadata-persistence.zod` copies stay as the single source. Why the kernel side goes, and why this was worth removing rather than living with: - Zero consumers. Import-statement scans across this repo, `cloud` and `objectui` find every consumer on `./system` (or `./contracts`' own interface); only `kernel/metadata-loader.test.ts` ever parsed the kernel copies. ADR-0049 enforce-or-remove. - The naming intuition pointed the wrong way, which is what made this sharper than an ordinary duplicate. The kernel copies were the ones that LOOKED canonical — normalized enums, required fields, a `.describe()` per property — and they were the dead ones; the live copy is the loose superset its own consumer calls "legacy". Picking by name, or by which reads as more rigorous, picked the dead one, and because the shapes overlap heavily that choice compiled and failed later, at an edge value (`add` vs `added`) or on a field one copy made required. No tombstone and no ADR-0087 conversion, deliberately: these are runtime envelope types, not authorable metadata, so no authored source can carry them and there is nothing for `os migrate meta` to rewrite (the plugin-runtime / dev-plugin precedents). `MetadataManagerConfig` and `MetadataFallbackStrategy` are untouched — they were never duplicated (kernel owns them, system re-exports them), and that is the split that survives: manager wiring is kernel's, the loader/watch envelope is system's, nothing is declared twice. `MetadataManagerConfig.formats` now reads the `shared` format enum (same four members, leaf module, no cycle) rather than a fourth local copy. Also: - `contracts/metadata-service.ts` drops the "spec carries TWO types named MetadataWatchEvent" warning added in #4404 — it no longer does. - `expression-conformance.ledger.ts` drops the now-absent `kernel/metadata-loader.zod.ts:filter` CEL surface (the surviving system options never declared a `filter`, so no loader predicate was ever evaluated through it). - Baselines dropped deliberately: `json-schema.manifest.json` −11 entries, `authorable-surface.json` −65 lines (nothing can author these, so no `[RETIRED]` markers). `api-surface.json` regenerated: 22 exports leave `./kernel`. `references/kernel/metadata-persistence.mdx` removed by `gen:docs`. v17 release notes + upgrade checklist extended. No runtime behaviour changes — nothing read the removed copies. The system shapes are NOT tightened here; narrowing them would be a separate change. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01M9uWvoEp9CoLzYjNExj9sL --- ...kernel-metadata-loader-envelope-removed.md | 78 +++ content/docs/references/kernel/meta.json | 1 - .../references/kernel/metadata-loader.mdx | 10 +- .../kernel/metadata-persistence.mdx | 200 -------- .../references/kernel/metadata-plugin.mdx | 4 +- .../references/system/metadata-loader.mdx | 2 +- content/docs/releases/v17.mdx | 12 +- .../test/expression-conformance.ledger.ts | 7 +- packages/spec/api-surface.json | 22 - packages/spec/authorable-surface.json | 67 +-- packages/spec/json-schema.manifest.json | 13 +- .../spec/src/contracts/metadata-service.ts | 11 +- .../spec/src/kernel/metadata-loader.test.ts | 414 +--------------- .../spec/src/kernel/metadata-loader.zod.ts | 448 ++---------------- .../spec/src/kernel/metadata-plugin.zod.ts | 4 +- .../src/system/metadata-persistence.zod.ts | 13 + .../objectstack-platform/references/_index.md | 3 +- 17 files changed, 173 insertions(+), 1136 deletions(-) create mode 100644 .changeset/kernel-metadata-loader-envelope-removed.md delete mode 100644 content/docs/references/kernel/metadata-persistence.mdx diff --git a/.changeset/kernel-metadata-loader-envelope-removed.md b/.changeset/kernel-metadata-loader-envelope-removed.md new file mode 100644 index 0000000000..3c8c1095c5 --- /dev/null +++ b/.changeset/kernel-metadata-loader-envelope-removed.md @@ -0,0 +1,78 @@ +--- +"@objectstack/spec": major +--- + +refactor(spec)!: remove the `kernel` metadata-loader envelope family — eleven names that each existed twice, with different shapes, on two subpath entries (#4411) + +`MetadataFormat`, `MetadataStats`, `MetadataLoadOptions`, `MetadataSaveOptions`, +`MetadataExportOptions`, `MetadataImportOptions`, `MetadataLoadResult`, +`MetadataSaveResult`, `MetadataWatchEvent`, `MetadataCollectionInfo` and +`MetadataLoaderContract` (plus each one's `…Schema`) are removed from +`@objectstack/spec/kernel` (`kernel/metadata-loader.zod`). Every one of those +names *also* existed, with a **different shape**, in +`@objectstack/spec/system` (`system/metadata-persistence.zod`). + +Which type you got depended on nothing but your import path: + +```ts +import type { MetadataWatchEvent } from '@objectstack/spec/kernel'; // one shape +import type { MetadataWatchEvent } from '@objectstack/spec/system'; // another +``` + +- **The `kernel` copies had zero consumers.** Import-statement scans across this + repo, `cloud` and `objectui` found every consumer importing from + `./system` (or, for the export/import options, `./contracts`' own interface). + Nothing but `kernel/metadata-loader.test.ts` ever parsed the `kernel` copies. +- **The naming intuition pointed the wrong way**, which is what made this worse + than an ordinary duplicate. The `kernel` copies were the ones that *looked* + canonical — normalized enums, required fields, a `.describe()` on every + property — and they were the dead ones. The live copy is the loose superset, + and `metadata-manager.ts` calls it "legacy" in its own comments. An + auto-import or a model completion picking by name, or by which one reads as + more rigorous, picked the dead one; because the shapes overlap heavily, that + choice compiled and only failed later, at an edge value (`add` vs `added`) or + on a field one copy made required. +- **No load path parsed them.** These are runtime envelope types, not authorable + metadata — no authored source can carry them. So there is deliberately **no** + `retiredKey()` tombstone and **no** ADR-0087 conversion: a prescription nobody + can receive is noise, and there is nothing for `os migrate meta` to rewrite + (the `plugin-runtime.zod.ts` / dev-plugin precedents, #3950, #4149). + +**FROM → TO — change the import path, keep the name:** + +```diff +-import type { MetadataWatchEvent, MetadataStats } from '@objectstack/spec/kernel'; ++import type { MetadataWatchEvent, MetadataStats } from '@objectstack/spec/system'; +``` + +The surviving `system` copy is the **looser** of the two, so a *reader* of these +types may need narrowing it did not need before; a *producer* needs nothing. The +differences that actually bite: + +| Type | `kernel` (removed) | `system` (keep) | +| --- | --- | --- | +| `MetadataWatchEvent.type` | `'added' \| 'changed' \| 'deleted'` | also `'add' \| 'change' \| 'unlink'` — the raw watcher values the runtime really emits | +| `MetadataWatchEvent` | `metadataType` / `name` / `timestamp` required | all three optional; adds `stats` | +| `MetadataStats` | `size` / `modifiedAt` / `etag` / `format` required | all optional; adds `mtime`, `hash` | +| `MetadataFormat` | `json \| yaml \| typescript \| javascript` | also the `yml` / `ts` / `js` aliases | +| `MetadataSaveResult.path` | required | optional; adds `stats` | +| `MetadataImportOptions` | `conflictResolution` / `dryRun` / `continueOnError` / `transform` | `source` / `strategy` / `validate` | +| `MetadataCollectionInfo` | `formats: MetadataFormat[]` | `namespaces: string[]` | + +No runtime behaviour changes: nothing read the removed copies. The `system` +shapes are **not** tightened here — they describe what `MetadataManager` +actually emits, and narrowing them would be a separate behaviour change. + +`MetadataManagerConfig` and `MetadataFallbackStrategy` are **unaffected**. They +were never duplicated — `kernel` owns them and `system` re-exports them — and +that is the split that survives: manager *wiring* is kernel's, the loader/watch +*envelope* is system's, and nothing is declared twice. + +The retirement kit: baselines dropped deliberately +(`json-schema.manifest.json` minus the 11 `kernel/Metadata*` entries; +`authorable-surface.json` minus the 65 matching lines — nothing can author +these, so no `[RETIRED]` markers); `api-surface.json` regenerated (22 exports +leave `./kernel`); `references/kernel/metadata-persistence.mdx` removed by +`gen:docs`; v17 release notes' dead-clusters table and upgrade checklist +extended. No liveness-ledger entries existed (the ledger tracks authorable +metadata types; these were never one). diff --git a/content/docs/references/kernel/meta.json b/content/docs/references/kernel/meta.json index a44766a69c..86bf413584 100644 --- a/content/docs/references/kernel/meta.json +++ b/content/docs/references/kernel/meta.json @@ -27,7 +27,6 @@ "execution-context", "metadata-customization", "metadata-loader", - "metadata-persistence", "metadata-plugin", "metadata-protection", "misc", diff --git a/content/docs/references/kernel/metadata-loader.mdx b/content/docs/references/kernel/metadata-loader.mdx index 1785ddb868..06aa9caf3f 100644 --- a/content/docs/references/kernel/metadata-loader.mdx +++ b/content/docs/references/kernel/metadata-loader.mdx @@ -5,13 +5,11 @@ description: Metadata Loader protocol schemas {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} -# Metadata Loader Protocol +# Metadata Manager Configuration -Defines the standard interface for loading and saving metadata in ObjectStack. +How the runtime `MetadataManager` is wired: which datasource backs `sys_metadata`, what to fall back to when that datasource is unreachable, cache / watch / validation settings, and the persistence write gates. -This protocol enables consistent metadata operations across different storage backends - -(filesystem, HTTP, S3, databases) and serialization formats (JSON, YAML, TypeScript). +The loader and watch *envelope* types (`MetadataFormat`, `MetadataStats`, `MetadataLoadOptions`, `MetadataWatchEvent`, `MetadataLoaderContract`, …) are NOT here — they live in `@objectstack/spec/system` (`system/metadata-persistence.zod`), which is their single source. **Source:** `packages/spec/src/kernel/metadata-loader.zod.ts` @@ -50,7 +48,7 @@ const result = MetadataFallbackStrategy.parse(data); | **tableName** | `string` | ✅ | Database table name for metadata storage | | **fallback** | `Enum<'filesystem' \| 'memory' \| 'none'>` | ✅ | Fallback strategy when datasource is unavailable | | **rootDir** | `string` | optional | Root directory path | -| **formats** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>[]` | ✅ | Enabled formats | +| **formats** | `Enum<'yaml' \| 'json' \| 'typescript' \| 'javascript'>[]` | ✅ | Enabled formats | | **cache** | `{ enabled: boolean; ttl: integer; maxSize?: integer; databaseLoader?: object }` | optional | Cache settings | | **watch** | `boolean` | ✅ | Enable file watching | | **watchOptions** | `{ ignored?: string[]; persistent: boolean; ignoreInitial: boolean }` | optional | File watcher options | diff --git a/content/docs/references/kernel/metadata-persistence.mdx b/content/docs/references/kernel/metadata-persistence.mdx deleted file mode 100644 index 9c6c423978..0000000000 --- a/content/docs/references/kernel/metadata-persistence.mdx +++ /dev/null @@ -1,200 +0,0 @@ ---- -title: Metadata Persistence -description: Metadata Persistence protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - - -**Source:** `packages/spec/src/kernel/metadata-persistence.zod.ts` - - -## TypeScript Usage - -```typescript -import { MetadataCollectionInfo, MetadataExportOptions, MetadataFormat, MetadataImportOptions, MetadataLoadOptions, MetadataLoadResult, MetadataLoaderContract, MetadataSaveOptions, MetadataSaveResult, MetadataStats, MetadataWatchEvent } from '@objectstack/spec/kernel'; -import type { MetadataCollectionInfo, MetadataExportOptions, MetadataFormat, MetadataImportOptions, MetadataLoadOptions, MetadataLoadResult, MetadataLoaderContract, MetadataSaveOptions, MetadataSaveResult, MetadataStats, MetadataWatchEvent } from '@objectstack/spec/kernel'; - -// Validate data -const result = MetadataCollectionInfo.parse(data); -``` - ---- - -## MetadataCollectionInfo - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `string` | ✅ | Collection type | -| **count** | `integer` | ✅ | Number of items | -| **formats** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>[]` | ✅ | Formats in collection | -| **totalSize** | `integer` | optional | Total size in bytes | -| **lastModified** | `string` | optional | Last modification date | -| **location** | `string` | optional | Collection location | - - ---- - -## MetadataExportOptions - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **output** | `string` | ✅ | Output file path | -| **format** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>` | optional | Export format | -| **filter** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | Filter items to export (CEL) | -| **includeStats** | `boolean` | optional | Include metadata statistics | -| **compress** | `boolean` | optional | Compress output (gzip) | -| **prettify** | `boolean` | optional | Pretty print output | - - ---- - -## MetadataFormat - -### Allowed Values - -* `json` -* `yaml` -* `typescript` -* `javascript` - - ---- - -## MetadataImportOptions - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **conflictResolution** | `Enum<'skip' \| 'overwrite' \| 'merge' \| 'fail'>` | ✅ | How to handle existing items | -| **validate** | `boolean` | ✅ | Validate before import | -| **dryRun** | `boolean` | ✅ | Simulate import without saving | -| **continueOnError** | `boolean` | ✅ | Continue if validation fails | -| **transform** | `string` | optional | Transform items before import | - - ---- - -## MetadataLoadOptions - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **patterns** | `string[]` | optional | File glob patterns | -| **ifNoneMatch** | `string` | optional | ETag for conditional request | -| **ifModifiedSince** | `string` | optional | Only load if modified after this date | -| **validate** | `boolean` | optional | Validate against schema | -| **useCache** | `boolean` | optional | Enable caching | -| **filter** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | optional | Filter predicate (CEL) | -| **limit** | `integer` | optional | Maximum items to load | -| **recursive** | `boolean` | optional | Search subdirectories | - - ---- - -## MetadataLoadResult - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **data** | `any \| null` | ✅ | Loaded metadata | -| **fromCache** | `boolean` | ✅ | Loaded from cache | -| **notModified** | `boolean` | ✅ | Not modified since last request | -| **etag** | `string` | optional | Entity tag | -| **stats** | `{ size: integer; modifiedAt: string; etag: string; format: Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>; … }` | optional | Metadata statistics | -| **loadTime** | `number` | optional | Load duration in ms | - - ---- - -## MetadataLoaderContract - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **name** | `string` | ✅ | Loader identifier | -| **protocol** | `Enum<'file:' \| 'http:' \| 's3:' \| 'datasource:' \| 'memory:'>` | ✅ | Protocol identifier | -| **capabilities** | `{ read: boolean; write: boolean; watch: boolean; list: boolean }` | ✅ | Loader capabilities | -| **supportedFormats** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>[]` | ✅ | Supported formats | -| **supportsWatch** | `boolean` | ✅ | Supports file watching | -| **supportsWrite** | `boolean` | ✅ | Supports write operations | -| **supportsCache** | `boolean` | ✅ | Supports caching | - - ---- - -## MetadataSaveOptions - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **format** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>` | ✅ | Output format | -| **prettify** | `boolean` | ✅ | Format with indentation | -| **indent** | `integer` | ✅ | Indentation spaces | -| **sortKeys** | `boolean` | ✅ | Sort object keys | -| **includeDefaults** | `boolean` | ✅ | Include default values | -| **backup** | `boolean` | ✅ | Create backup file | -| **overwrite** | `boolean` | ✅ | Overwrite existing file | -| **atomic** | `boolean` | ✅ | Use atomic write operation | -| **path** | `string` | optional | Custom output path | - - ---- - -## MetadataSaveResult - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **success** | `boolean` | ✅ | Save successful | -| **path** | `string` | ✅ | Output path | -| **etag** | `string` | optional | Generated entity tag | -| **size** | `integer` | optional | File size | -| **saveTime** | `number` | optional | Save duration in ms | -| **backupPath** | `string` | optional | Backup file path | - - ---- - -## MetadataStats - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **size** | `integer` | ✅ | File size in bytes | -| **modifiedAt** | `string` | ✅ | Last modified date | -| **etag** | `string` | ✅ | Entity tag for cache validation | -| **format** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>` | ✅ | Serialization format | -| **path** | `string` | optional | File system path | -| **metadata** | `Record` | optional | Provider-specific metadata | - - ---- - -## MetadataWatchEvent - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `Enum<'added' \| 'changed' \| 'deleted'>` | ✅ | Event type | -| **metadataType** | `string` | ✅ | Type of metadata | -| **name** | `string` | ✅ | Item identifier | -| **path** | `string` | ✅ | File path | -| **data** | `any` | optional | Item data | -| **timestamp** | `string` | ✅ | Event timestamp | - - ---- - diff --git a/content/docs/references/kernel/metadata-plugin.mdx b/content/docs/references/kernel/metadata-plugin.mdx index 976eea9b4a..84e932554f 100644 --- a/content/docs/references/kernel/metadata-plugin.mdx +++ b/content/docs/references/kernel/metadata-plugin.mdx @@ -57,11 +57,11 @@ cohesive plugin that "takes over" the entire platform's metadata management: ## References -- [kernel/metadata-loader.zod.ts](/docs/references/kernel/metadata-loader) — Storage backend protocol +- [kernel/metadata-loader.zod.ts](/docs/references/kernel/metadata-loader) — MetadataManager wiring (datasource, cache, write gates) - [kernel/metadata-customization.zod.ts](/docs/references/kernel/metadata-customization) — Overlay/merge protocol -- [system/metadata-persistence.zod.ts](/docs/references/system/metadata-persistence) — Database record format +- [system/metadata-persistence.zod.ts](/docs/references/system/metadata-persistence) — Database record format + loader/watch envelope types - contracts/metadata-service.ts — Service interface diff --git a/content/docs/references/system/metadata-loader.mdx b/content/docs/references/system/metadata-loader.mdx index 43bc2a5c7d..369a209237 100644 --- a/content/docs/references/system/metadata-loader.mdx +++ b/content/docs/references/system/metadata-loader.mdx @@ -42,7 +42,7 @@ const result = MetadataFallbackStrategy.parse(data); | **tableName** | `string` | ✅ | Database table name for metadata storage | | **fallback** | `Enum<'filesystem' \| 'memory' \| 'none'>` | ✅ | Fallback strategy when datasource is unavailable | | **rootDir** | `string` | optional | Root directory path | -| **formats** | `Enum<'json' \| 'yaml' \| 'typescript' \| 'javascript'>[]` | ✅ | Enabled formats | +| **formats** | `Enum<'yaml' \| 'json' \| 'typescript' \| 'javascript'>[]` | ✅ | Enabled formats | | **cache** | `{ enabled: boolean; ttl: integer; maxSize?: integer; databaseLoader?: object }` | optional | Cache settings | | **watch** | `boolean` | ✅ | Enable file watching | | **watchOptions** | `{ ignored?: string[]; persistent: boolean; ignoreInitial: boolean }` | optional | File watcher options | diff --git a/content/docs/releases/v17.mdx b/content/docs/releases/v17.mdx index 3c3ac3da59..e1da2ecf0f 100644 --- a/content/docs/releases/v17.mdx +++ b/content/docs/releases/v17.mdx @@ -970,6 +970,7 @@ import or the authored key. | `DEFAULT_DISPATCHER_ROUTES` | dead route table | | Aspirational config on Theme / Translation / Webhook | still-dead after #3494 | | `ChartInteraction.zoom` / `.clickAction` | never implemented (#3752) | +| The `kernel` metadata-loader envelope family — `MetadataFormat`, `MetadataStats`, `MetadataLoadOptions`, `MetadataSaveOptions`, `MetadataExportOptions`, `MetadataImportOptions`, `MetadataLoadResult`, `MetadataSaveResult`, `MetadataWatchEvent`, `MetadataCollectionInfo`, `MetadataLoaderContract` (`@objectstack/spec/kernel`) | eleven names that each existed **twice**, with a different shape, on `./kernel` and `./system` — so which type you got depended on your import path. Every consumer imported the `./system` copy; the `./kernel` copies had zero consumers. Import them from `@objectstack/spec/system` (#4411, ADR-0049). `MetadataManagerConfig` / `MetadataFallbackStrategy` are unaffected and still ship from both entries | The Console side follows: `@object-ui/types` drops its `ObjectStack`/`ObjectOS`/`ObjectQL`/`ObjectUI` Capabilities re-exports, which @@ -2032,7 +2033,16 @@ covers are folded into the list below rather than left to the changelog.) `maplibre-gl` 5→6 / `chalk` 5→6 major bumps. - **Type importers:** replace `ObjectStackProtocol` / `ObjectStackProtocolSchema` with the narrowest per-domain slices; drop GraphQL types and any of the removed - dead spec clusters. + dead spec clusters. If you imported `MetadataFormat`, `MetadataStats`, + `MetadataLoadOptions`, `MetadataSaveOptions`, `MetadataExportOptions`, + `MetadataImportOptions`, `MetadataLoadResult`, `MetadataSaveResult`, + `MetadataWatchEvent`, `MetadataCollectionInfo` or `MetadataLoaderContract` + from `@objectstack/spec/kernel`, change the path to `@objectstack/spec/system` + — same names, and that copy is the one the runtime has always emitted. It is + the *looser* of the two, so a reader may need new narrowing: notably + `MetadataWatchEvent.type` also carries the raw watcher values + `add`/`change`/`unlink`, and `metadataType`/`name`/`timestamp` are optional + there. Nothing to migrate at runtime — the values were always these. - **Multi-org:** the `group` posture requires the enterprise runtime — deployments relying on it self-activating must install `@objectstack/organizations` or move to `isolated`. diff --git a/packages/qa/dogfood/test/expression-conformance.ledger.ts b/packages/qa/dogfood/test/expression-conformance.ledger.ts index 3abf3d9678..bf7ff0b0f2 100644 --- a/packages/qa/dogfood/test/expression-conformance.ledger.ts +++ b/packages/qa/dogfood/test/expression-conformance.ledger.ts @@ -155,7 +155,12 @@ export const EXPRESSION_SURFACE: ExprSurface[] = [ covers: [ 'automation/flow.zod.ts:condition', 'automation/sync.zod.ts:condition', - 'kernel/metadata-loader.zod.ts:filter', + // `kernel/metadata-loader.zod.ts:filter` (on MetadataLoadOptions and + // MetadataExportOptions) was removed with the rest of that file's + // zero-consumer duplicate envelope family in #4411. The surviving + // `system/metadata-persistence.zod` copies of those options never + // declared a `filter` — so no loader predicate was ever evaluated + // through this surface, and there is nothing to re-point at. ], }, { diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index c24fc55e53..7c00b2d93e 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -1563,28 +1563,14 @@ "MetadataChangeTypeSchema (const)", "MetadataChangedEventPayload (type)", "MetadataChangedEventPayloadSchema (const)", - "MetadataCollectionInfo (type)", - "MetadataCollectionInfoSchema (const)", "MetadataDependency (type)", "MetadataDependencySchema (const)", "MetadataDiffItem (type)", "MetadataDiffItemSchema (const)", "MetadataEvent (type)", "MetadataEventSchema (const)", - "MetadataExportOptions (type)", - "MetadataExportOptionsSchema (const)", "MetadataFallbackStrategy (type)", "MetadataFallbackStrategySchema (const)", - "MetadataFormat (type)", - "MetadataFormatSchema (const)", - "MetadataImportOptions (type)", - "MetadataImportOptionsSchema (const)", - "MetadataLoadOptions (type)", - "MetadataLoadOptionsSchema (const)", - "MetadataLoadResult (type)", - "MetadataLoadResultSchema (const)", - "MetadataLoaderContract (type)", - "MetadataLoaderContractSchema (const)", "MetadataLock (type)", "MetadataLockSchema (const)", "MetadataLockSource (type)", @@ -1605,20 +1591,12 @@ "MetadataQueryResultSchema (const)", "MetadataQuerySchema (const)", "MetadataReadDecoration (type)", - "MetadataSaveOptions (type)", - "MetadataSaveOptionsSchema (const)", - "MetadataSaveResult (type)", - "MetadataSaveResultSchema (const)", - "MetadataStats (type)", - "MetadataStatsSchema (const)", "MetadataType (type)", "MetadataTypeRegistryEntry (type)", "MetadataTypeRegistryEntrySchema (const)", "MetadataTypeSchema (const)", "MetadataValidationResult (type)", "MetadataValidationResultSchema (const)", - "MetadataWatchEvent (type)", - "MetadataWatchEventSchema (const)", "MultiVersionSupport (type)", "MultiVersionSupportSchema (const)", "NamespaceConflictError (type)", diff --git a/packages/spec/authorable-surface.json b/packages/spec/authorable-surface.json index fb905f2dd6..f28d5bf1f1 100644 --- a/packages/spec/authorable-surface.json +++ b/packages/spec/authorable-surface.json @@ -1,5 +1,5 @@ { - "description": "Ratchet of every AUTHORABLE key in the spec — what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 §5.", + "description": "Ratchet of every AUTHORABLE key in the spec \u2014 what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 \u00a75.", "keys": [ "ai/AIModelConfig:maxTokens", "ai/AIModelConfig:model", @@ -4992,12 +4992,6 @@ "kernel/MetadataBulkResult:failed", "kernel/MetadataBulkResult:succeeded", "kernel/MetadataBulkResult:total", - "kernel/MetadataCollectionInfo:count", - "kernel/MetadataCollectionInfo:formats", - "kernel/MetadataCollectionInfo:lastModified", - "kernel/MetadataCollectionInfo:location", - "kernel/MetadataCollectionInfo:totalSize", - "kernel/MetadataCollectionInfo:type", "kernel/MetadataDependency:kind", "kernel/MetadataDependency:sourceName", "kernel/MetadataDependency:sourceType", @@ -5017,38 +5011,6 @@ "kernel/MetadataEvent:packageId", "kernel/MetadataEvent:payload", "kernel/MetadataEvent:timestamp", - "kernel/MetadataExportOptions:compress", - "kernel/MetadataExportOptions:filter", - "kernel/MetadataExportOptions:format", - "kernel/MetadataExportOptions:includeStats", - "kernel/MetadataExportOptions:output", - "kernel/MetadataExportOptions:prettify", - "kernel/MetadataImportOptions:conflictResolution", - "kernel/MetadataImportOptions:continueOnError", - "kernel/MetadataImportOptions:dryRun", - "kernel/MetadataImportOptions:transform", - "kernel/MetadataImportOptions:validate", - "kernel/MetadataLoadOptions:filter", - "kernel/MetadataLoadOptions:ifModifiedSince", - "kernel/MetadataLoadOptions:ifNoneMatch", - "kernel/MetadataLoadOptions:limit", - "kernel/MetadataLoadOptions:patterns", - "kernel/MetadataLoadOptions:recursive", - "kernel/MetadataLoadOptions:useCache", - "kernel/MetadataLoadOptions:validate", - "kernel/MetadataLoadResult:data", - "kernel/MetadataLoadResult:etag", - "kernel/MetadataLoadResult:fromCache", - "kernel/MetadataLoadResult:loadTime", - "kernel/MetadataLoadResult:notModified", - "kernel/MetadataLoadResult:stats", - "kernel/MetadataLoaderContract:capabilities", - "kernel/MetadataLoaderContract:name", - "kernel/MetadataLoaderContract:protocol", - "kernel/MetadataLoaderContract:supportedFormats", - "kernel/MetadataLoaderContract:supportsCache", - "kernel/MetadataLoaderContract:supportsWatch", - "kernel/MetadataLoaderContract:supportsWrite", "kernel/MetadataManagerConfig:cache", "kernel/MetadataManagerConfig:datasource", "kernel/MetadataManagerConfig:fallback", @@ -5106,27 +5068,6 @@ "kernel/MetadataQueryResult:page", "kernel/MetadataQueryResult:pageSize", "kernel/MetadataQueryResult:total", - "kernel/MetadataSaveOptions:atomic", - "kernel/MetadataSaveOptions:backup", - "kernel/MetadataSaveOptions:format", - "kernel/MetadataSaveOptions:includeDefaults", - "kernel/MetadataSaveOptions:indent", - "kernel/MetadataSaveOptions:overwrite", - "kernel/MetadataSaveOptions:path", - "kernel/MetadataSaveOptions:prettify", - "kernel/MetadataSaveOptions:sortKeys", - "kernel/MetadataSaveResult:backupPath", - "kernel/MetadataSaveResult:etag", - "kernel/MetadataSaveResult:path", - "kernel/MetadataSaveResult:saveTime", - "kernel/MetadataSaveResult:size", - "kernel/MetadataSaveResult:success", - "kernel/MetadataStats:etag", - "kernel/MetadataStats:format", - "kernel/MetadataStats:metadata", - "kernel/MetadataStats:modifiedAt", - "kernel/MetadataStats:path", - "kernel/MetadataStats:size", "kernel/MetadataTypeRegistryEntry:actions", "kernel/MetadataTypeRegistryEntry:allowOrgOverride", "kernel/MetadataTypeRegistryEntry:allowRuntimeCreate", @@ -5142,12 +5083,6 @@ "kernel/MetadataValidationResult:errors", "kernel/MetadataValidationResult:valid", "kernel/MetadataValidationResult:warnings", - "kernel/MetadataWatchEvent:data", - "kernel/MetadataWatchEvent:metadataType", - "kernel/MetadataWatchEvent:name", - "kernel/MetadataWatchEvent:path", - "kernel/MetadataWatchEvent:timestamp", - "kernel/MetadataWatchEvent:type", "kernel/MultiVersionSupport:enabled", "kernel/MultiVersionSupport:maxConcurrentVersions", "kernel/MultiVersionSupport:rollout", diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index 4022e4eb8d..e3e37a07bf 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -1,5 +1,5 @@ { - "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema — remove a key ONLY for a deliberate retirement. See #2978.", + "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema \u2014 remove a key ONLY for a deliberate retirement. See #2978.", "schemas": [ "ai/AIModelConfig", "ai/AIUsageRecord", @@ -1025,17 +1025,10 @@ "kernel/MetadataCategoryEnum", "kernel/MetadataChangeOperation", "kernel/MetadataChangeType", - "kernel/MetadataCollectionInfo", "kernel/MetadataDependency", "kernel/MetadataDiffItem", "kernel/MetadataEvent", - "kernel/MetadataExportOptions", "kernel/MetadataFallbackStrategy", - "kernel/MetadataFormat", - "kernel/MetadataImportOptions", - "kernel/MetadataLoadOptions", - "kernel/MetadataLoadResult", - "kernel/MetadataLoaderContract", "kernel/MetadataLock", "kernel/MetadataLockSource", "kernel/MetadataManagerConfig", @@ -1045,13 +1038,9 @@ "kernel/MetadataProvenance", "kernel/MetadataQuery", "kernel/MetadataQueryResult", - "kernel/MetadataSaveOptions", - "kernel/MetadataSaveResult", - "kernel/MetadataStats", "kernel/MetadataType", "kernel/MetadataTypeRegistryEntry", "kernel/MetadataValidationResult", - "kernel/MetadataWatchEvent", "kernel/MultiVersionSupport", "kernel/NamespaceConflictError", "kernel/NamespaceRegistryEntry", diff --git a/packages/spec/src/contracts/metadata-service.ts b/packages/spec/src/contracts/metadata-service.ts index 1af6394adb..62fc8051f8 100644 --- a/packages/spec/src/contracts/metadata-service.ts +++ b/packages/spec/src/contracts/metadata-service.ts @@ -37,12 +37,11 @@ import type { MetadataQuery, MetadataQueryResult, MetadataValidationResult, MetadataBulkResult, MetadataDependency } from '../kernel/metadata-plugin.zod'; // The PERSISTENCE-side watch event (`add`/`added`/`changed`/`deleted`/…, path + -// file stats) — what `MetadataManager.subscribe` actually relays. NOT the -// near-namesake in `../kernel/metadata-loader.zod`: spec carries TWO types -// named `MetadataWatchEvent` with different shapes (reported on #4251; merging -// them is its own change), and `MetadataManager implements IMetadataService` -// rejected the first draft of this import — which is exactly the check doing -// its job. +// file stats) — what `MetadataManager.subscribe` relays, as opposed to the +// registration-level events `watch` forwards (`MetadataWatchCallback` below). +// Spec used to carry a second, differently-shaped `MetadataWatchEvent` on +// `@objectstack/spec/kernel`; it had no consumers and was removed in #4411, so +// this is now the only type by that name. import type { MetadataWatchEvent } from '../system/metadata-persistence.zod'; import type { Action } from '../ui/action.zod'; import type { MetadataOverlay } from '../kernel/metadata-customization.zod'; diff --git a/packages/spec/src/kernel/metadata-loader.test.ts b/packages/spec/src/kernel/metadata-loader.test.ts index f885667fe8..c94e10c48a 100644 --- a/packages/spec/src/kernel/metadata-loader.test.ts +++ b/packages/spec/src/kernel/metadata-loader.test.ts @@ -1,403 +1,25 @@ import { describe, it, expect } from 'vitest'; import { - MetadataFormatSchema, - MetadataStatsSchema, - MetadataLoadOptionsSchema, - MetadataSaveOptionsSchema, - MetadataExportOptionsSchema, - MetadataImportOptionsSchema, - MetadataLoadResultSchema, - MetadataSaveResultSchema, - MetadataWatchEventSchema, - MetadataCollectionInfoSchema, - MetadataLoaderContractSchema, + MetadataFallbackStrategySchema, MetadataManagerConfigSchema, } from './metadata-loader.zod'; -describe('MetadataLoaderProtocol', () => { - describe('MetadataFormatSchema', () => { - it('should accept valid formats', () => { - expect(MetadataFormatSchema.parse('json')).toBe('json'); - expect(MetadataFormatSchema.parse('yaml')).toBe('yaml'); - expect(MetadataFormatSchema.parse('typescript')).toBe('typescript'); - expect(MetadataFormatSchema.parse('javascript')).toBe('javascript'); +// The loader/persistence envelope vocabulary this file used to also cover +// (`MetadataFormat`, `MetadataStats`, `MetadataLoad*`, `MetadataSave*`, +// `MetadataExport/ImportOptions`, `MetadataWatchEvent`, +// `MetadataCollectionInfo`, `MetadataLoaderContract`) was a zero-consumer +// duplicate of `system/metadata-persistence.zod`, removed in #4411. Its tests +// live with the surviving source: `../system/metadata-persistence.test.ts`. +describe('MetadataManagerConfig', () => { + describe('MetadataFallbackStrategySchema', () => { + it('should accept every fallback strategy', () => { + for (const strategy of ['filesystem', 'memory', 'none'] as const) { + expect(MetadataFallbackStrategySchema.parse(strategy)).toBe(strategy); + } }); - it('should reject invalid formats', () => { - expect(() => MetadataFormatSchema.parse('xml')).toThrow(); - expect(() => MetadataFormatSchema.parse('toml')).toThrow(); - }); - }); - - describe('MetadataStatsSchema', () => { - it('should validate metadata statistics', () => { - const stats = { - size: 1024, - modifiedAt: '2026-01-31T00:00:00.000Z', - etag: '"abc123"', - format: 'json' as const, - }; - - const result = MetadataStatsSchema.parse(stats); - expect(result.size).toBe(1024); - expect(result.etag).toBe('"abc123"'); - expect(result.format).toBe('json'); - }); - - it('should allow optional fields', () => { - const stats = { - size: 2048, - modifiedAt: new Date().toISOString(), - etag: '"xyz789"', - format: 'yaml' as const, - path: '/metadata/objects/customer.object.yaml', - metadata: { encoding: 'utf-8' }, - }; - - const result = MetadataStatsSchema.parse(stats); - expect(result.path).toBe('/metadata/objects/customer.object.yaml'); - expect(result.metadata).toEqual({ encoding: 'utf-8' }); - }); - - it('should reject negative size', () => { - const stats = { - size: -100, - modifiedAt: new Date().toISOString(), - etag: '"abc"', - format: 'json' as const, - }; - - expect(() => MetadataStatsSchema.parse(stats)).toThrow(); - }); - }); - - describe('MetadataLoadOptionsSchema', () => { - it('should apply default values', () => { - const options = {}; - const result = MetadataLoadOptionsSchema.parse(options); - - expect(result.validate).toBe(true); - expect(result.useCache).toBe(true); - expect(result.recursive).toBe(true); - }); - - it('should accept all options', () => { - const options = { - patterns: ['**/*.object.ts', '**/*.object.json'], - ifNoneMatch: '"etag123"', - ifModifiedSince: '2026-01-01T00:00:00.000Z', - validate: false, - useCache: false, - filter: '(item) => item.name.startsWith("sys_")', - limit: 100, - recursive: false, - }; - - const result = MetadataLoadOptionsSchema.parse(options); - expect(result.patterns).toHaveLength(2); - expect(result.limit).toBe(100); - expect(result.validate).toBe(false); - }); - }); - - describe('MetadataSaveOptionsSchema', () => { - it('should apply default values', () => { - const options = {}; - const result = MetadataSaveOptionsSchema.parse(options); - - expect(result.format).toBe('typescript'); - expect(result.prettify).toBe(true); - expect(result.indent).toBe(2); - expect(result.overwrite).toBe(true); - expect(result.atomic).toBe(true); - }); - - it('should validate indent range', () => { - expect(() => - MetadataSaveOptionsSchema.parse({ indent: -1 }) - ).toThrow(); - - expect(() => - MetadataSaveOptionsSchema.parse({ indent: 10 }) - ).toThrow(); - - expect( - MetadataSaveOptionsSchema.parse({ indent: 4 }).indent - ).toBe(4); - }); - - it('should accept custom path', () => { - const options = { - path: '/custom/path/object.ts', - format: 'json' as const, - }; - - const result = MetadataSaveOptionsSchema.parse(options); - expect(result.path).toBe('/custom/path/object.ts'); - expect(result.format).toBe('json'); - }); - }); - - describe('MetadataExportOptionsSchema', () => { - it('should require output path', () => { - expect(() => MetadataExportOptionsSchema.parse({})).toThrow(); - - const options = { output: './export/objects.json' }; - const result = MetadataExportOptionsSchema.parse(options); - expect(result.output).toBe('./export/objects.json'); - }); - - it('should apply defaults', () => { - const options = { output: './export.json' }; - const result = MetadataExportOptionsSchema.parse(options); - - expect(result.format).toBe('json'); - expect(result.includeStats).toBe(false); - expect(result.compress).toBe(false); - expect(result.prettify).toBe(true); - }); - }); - - describe('MetadataImportOptionsSchema', () => { - it('should apply default conflict resolution', () => { - const options = {}; - const result = MetadataImportOptionsSchema.parse(options); - - expect(result.conflictResolution).toBe('merge'); - expect(result.validate).toBe(true); - expect(result.dryRun).toBe(false); - expect(result.continueOnError).toBe(false); - }); - - it('should accept all conflict strategies', () => { - const strategies = ['skip', 'overwrite', 'merge', 'fail'] as const; - - strategies.forEach(strategy => { - const result = MetadataImportOptionsSchema.parse({ - conflictResolution: strategy - }); - expect(result.conflictResolution).toBe(strategy); - }); - }); - - it('should accept transform function', () => { - const options = { - transform: '(item) => ({ ...item, imported: true })', - }; - - const result = MetadataImportOptionsSchema.parse(options); - expect(result.transform).toBeDefined(); - }); - }); - - describe('MetadataLoadResultSchema', () => { - it('should validate load result', () => { - const result = { - data: { name: 'customer', label: 'Customer' }, - fromCache: false, - notModified: false, - }; - - const validated = MetadataLoadResultSchema.parse(result); - expect(validated.data).toBeDefined(); - expect(validated.fromCache).toBe(false); - }); - - it('should accept null data (not found)', () => { - const result = { - data: null, - fromCache: false, - notModified: false, - }; - - const validated = MetadataLoadResultSchema.parse(result); - expect(validated.data).toBeNull(); - }); - - it('should include optional fields', () => { - const result = { - data: { name: 'test' }, - fromCache: true, - notModified: true, - etag: '"abc123"', - stats: { - size: 512, - modifiedAt: new Date().toISOString(), - etag: '"abc123"', - format: 'typescript' as const, - }, - loadTime: 45.5, - }; - - const validated = MetadataLoadResultSchema.parse(result); - expect(validated.etag).toBe('"abc123"'); - expect(validated.loadTime).toBe(45.5); - expect(validated.stats).toBeDefined(); - }); - }); - - describe('MetadataSaveResultSchema', () => { - it('should validate save result', () => { - const result = { - success: true, - path: '/metadata/objects/customer.object.ts', - }; - - const validated = MetadataSaveResultSchema.parse(result); - expect(validated.success).toBe(true); - expect(validated.path).toBeDefined(); - }); - - it('should include optional fields', () => { - const result = { - success: true, - path: '/metadata/objects/customer.object.ts', - etag: '"new-etag"', - size: 2048, - saveTime: 12.3, - backupPath: '/metadata/objects/customer.object.ts.bak', - }; - - const validated = MetadataSaveResultSchema.parse(result); - expect(validated.size).toBe(2048); - expect(validated.backupPath).toBeDefined(); - }); - }); - - describe('MetadataWatchEventSchema', () => { - it('should validate watch events', () => { - const events = [ - { - type: 'added' as const, - metadataType: 'object', - name: 'customer', - path: '/objects/customer.object.ts', - data: { name: 'customer' }, - timestamp: new Date().toISOString(), - }, - { - type: 'changed' as const, - metadataType: 'view', - name: 'customer_list', - path: '/views/customer_list.view.ts', - timestamp: new Date().toISOString(), - }, - { - type: 'deleted' as const, - metadataType: 'app', - name: 'old_app', - path: '/apps/old_app.ts', - timestamp: new Date().toISOString(), - }, - ]; - - events.forEach(event => { - const validated = MetadataWatchEventSchema.parse(event); - expect(validated.type).toBe(event.type); - expect(validated.metadataType).toBeDefined(); - }); - }); - }); - - describe('MetadataCollectionInfoSchema', () => { - it('should validate collection info', () => { - const info = { - type: 'object', - count: 42, - formats: ['typescript', 'json'] as const, - }; - - const validated = MetadataCollectionInfoSchema.parse(info); - expect(validated.count).toBe(42); - expect(validated.formats).toHaveLength(2); - }); - - it('should accept optional fields', () => { - const info = { - type: 'view', - count: 15, - formats: ['yaml'] as const, - totalSize: 51200, - lastModified: '2026-01-31T00:00:00.000Z', - location: '/metadata/views', - }; - - const validated = MetadataCollectionInfoSchema.parse(info); - expect(validated.totalSize).toBe(51200); - expect(validated.location).toBe('/metadata/views'); - }); - }); - - describe('MetadataLoaderContractSchema', () => { - it('should validate loader contract', () => { - const contract = { - name: 'filesystem', - protocol: 'file:', - capabilities: { - read: true, - write: true, - watch: false, - list: true, - }, - supportedFormats: ['json', 'yaml', 'typescript'] as const, - }; - - const validated = MetadataLoaderContractSchema.parse(contract); - expect(validated.name).toBe('filesystem'); - expect(validated.protocol).toBe('file:'); - expect(validated.supportsWatch).toBe(false); // default - expect(validated.supportsWrite).toBe(true); // default - expect(validated.supportsCache).toBe(true); // default - }); - - it('should allow custom capabilities', () => { - const contract = { - name: 'http', - protocol: 'http:', - capabilities: { - read: true, - write: false, - watch: false, - list: false, - }, - supportedFormats: ['json'] as const, - supportsWatch: false, - supportsWrite: false, - supportsCache: true, - }; - - const validated = MetadataLoaderContractSchema.parse(contract); - expect(validated.protocol).toBe('http:'); - expect(validated.supportsWrite).toBe(false); - expect(validated.supportsCache).toBe(true); - }); - - it('should accept datasource protocol', () => { - const contract = { - name: 'database', - protocol: 'datasource:', - capabilities: { read: true, write: true, watch: false, list: true }, - supportedFormats: ['json'] as const, - }; - - const validated = MetadataLoaderContractSchema.parse(contract); - expect(validated.protocol).toBe('datasource:'); - expect(validated.capabilities.write).toBe(true); - }); - - it('should accept all valid protocols', () => { - const protocols = ['file:', 'http:', 's3:', 'datasource:', 'memory:']; - protocols.forEach((protocol) => { - expect(() => MetadataLoaderContractSchema.parse({ - name: 'test', protocol, capabilities: {}, supportedFormats: ['json'], - })).not.toThrow(); - }); - }); - - it('should reject invalid protocol', () => { - expect(() => MetadataLoaderContractSchema.parse({ - name: 'test', protocol: 'ftp:', capabilities: {}, supportedFormats: ['json'], - })).toThrow(); + it('should reject an unknown strategy', () => { + expect(() => MetadataFallbackStrategySchema.parse('redis')).toThrow(); }); }); @@ -405,7 +27,7 @@ describe('MetadataLoaderProtocol', () => { it('should apply defaults', () => { const config = {}; const validated = MetadataManagerConfigSchema.parse(config); - + expect(validated.formats).toEqual(['typescript', 'json', 'yaml']); expect(validated.watch).toBe(false); expect(validated.tableName).toBe('sys_metadata'); @@ -452,7 +74,7 @@ describe('MetadataLoaderProtocol', () => { encoding: 'utf-8', }, }; - + const validated = MetadataManagerConfigSchema.parse(config); expect(validated.datasource).toBe('postgres_main'); expect(validated.rootDir).toBe('/metadata'); @@ -477,7 +99,7 @@ describe('MetadataLoaderProtocol', () => { const config = { cache: { enabled: true, ttl: -100 }, }; - + expect(() => MetadataManagerConfigSchema.parse(config)).toThrow(); }); }); diff --git a/packages/spec/src/kernel/metadata-loader.zod.ts b/packages/spec/src/kernel/metadata-loader.zod.ts index 0e3e6cebe3..fb02e526e0 100644 --- a/packages/spec/src/kernel/metadata-loader.zod.ts +++ b/packages/spec/src/kernel/metadata-loader.zod.ts @@ -3,411 +3,32 @@ import { z } from 'zod'; /** - * # Metadata Loader Protocol - * - * Defines the standard interface for loading and saving metadata in ObjectStack. - * This protocol enables consistent metadata operations across different storage backends - * (filesystem, HTTP, S3, databases) and serialization formats (JSON, YAML, TypeScript). + * # Metadata Manager Configuration + * + * How the runtime `MetadataManager` is wired: which datasource backs `sys_metadata`, what to fall back to when that datasource is unreachable, cache / watch / validation settings, and the persistence write gates. + * + * The loader and watch *envelope* types (`MetadataFormat`, `MetadataStats`, `MetadataLoadOptions`, `MetadataWatchEvent`, `MetadataLoaderContract`, …) are NOT here — they live in `@objectstack/spec/system` (`system/metadata-persistence.zod`), which is their single source. */ -/** - * Metadata Format Enum - * Supported serialization formats for metadata - */ -import { lazySchema } from '../shared/lazy-schema'; -import { ExpressionInputSchema } from '../shared/expression.zod'; -export const MetadataFormatSchema = lazySchema(() => z.enum(['json', 'yaml', 'typescript', 'javascript'])); - -/** - * Metadata Statistics - * Information about a metadata item without loading its full content - */ -export const MetadataStatsSchema = lazySchema(() => z.object({ - /** - * Size of the metadata file in bytes - */ - size: z.number().int().min(0).describe('File size in bytes'), - - /** - * Last modification timestamp - */ - modifiedAt: z.string().datetime().describe('Last modified date'), - - /** - * ETag for cache validation - * Used for conditional requests (If-None-Match header) - */ - etag: z.string().describe('Entity tag for cache validation'), - - /** - * Serialization format - */ - format: MetadataFormatSchema.describe('Serialization format'), - - /** - * Full file path (if applicable) - */ - path: z.string().optional().describe('File system path'), - - /** - * Additional metadata provider-specific properties - */ - metadata: z.record(z.string(), z.unknown()).optional().describe('Provider-specific metadata'), -})); - -/** - * Metadata Load Options - */ -export const MetadataLoadOptionsSchema = lazySchema(() => z.object({ - /** - * Glob patterns to match files - * Example: ["**\/*.object.ts", "**\/*.object.json"] - */ - patterns: z.array(z.string()).optional().describe('File glob patterns'), - - /** - * If-None-Match header for conditional loading - * Only load if ETag doesn't match - */ - ifNoneMatch: z.string().optional().describe('ETag for conditional request'), - - /** - * If-Modified-Since header for conditional loading - */ - ifModifiedSince: z.string().datetime().optional().describe('Only load if modified after this date'), - - /** - * Whether to validate against Zod schema - */ - validate: z.boolean().default(true).describe('Validate against schema'), - - /** - * Whether to use cache if available - */ - useCache: z.boolean().default(true).describe('Enable caching'), - - /** - * Filter predicate — CEL expression evaluated against each metadata item. - * Example: P`item.name.startsWith('sys_')` - */ - filter: ExpressionInputSchema.optional().describe('Filter predicate (CEL)'), - - /** - * Maximum number of items to load - */ - limit: z.number().int().min(1).optional().describe('Maximum items to load'), - - /** - * Recursively search subdirectories - */ - recursive: z.boolean().default(true).describe('Search subdirectories'), -})); - -/** - * Metadata Save Options - */ -export const MetadataSaveOptionsSchema = lazySchema(() => z.object({ - /** - * Serialization format - */ - format: MetadataFormatSchema.default('typescript').describe('Output format'), - - /** - * Prettify output (formatted with indentation) - */ - prettify: z.boolean().default(true).describe('Format with indentation'), - - /** - * Indentation size (spaces) - */ - indent: z.number().int().min(0).max(8).default(2).describe('Indentation spaces'), - - /** - * Sort object keys alphabetically - */ - sortKeys: z.boolean().default(false).describe('Sort object keys'), - - /** - * Include default values in output - */ - includeDefaults: z.boolean().default(false).describe('Include default values'), - - /** - * Create backup before overwriting - */ - backup: z.boolean().default(false).describe('Create backup file'), - - /** - * Overwrite if exists - */ - overwrite: z.boolean().default(true).describe('Overwrite existing file'), - - /** - * Atomic write (write to temp file, then rename) - */ - atomic: z.boolean().default(true).describe('Use atomic write operation'), - - /** - * Custom file path (overrides default location) - */ - path: z.string().optional().describe('Custom output path'), -})); - -/** - * Metadata Export Options - */ -export const MetadataExportOptionsSchema = lazySchema(() => z.object({ - /** - * Output file path - */ - output: z.string().describe('Output file path'), - - /** - * Export format - */ - format: MetadataFormatSchema.default('json').describe('Export format'), - - /** - * Filter predicate — CEL expression evaluated against each metadata item. - */ - filter: ExpressionInputSchema.optional().describe('Filter items to export (CEL)'), - - /** - * Include statistics in export - */ - includeStats: z.boolean().default(false).describe('Include metadata statistics'), - - /** - * Compress output - */ - compress: z.boolean().default(false).describe('Compress output (gzip)'), - - /** - * Pretty print output - */ - prettify: z.boolean().default(true).describe('Pretty print output'), -})); - -/** - * Metadata Import Options - */ -export const MetadataImportOptionsSchema = lazySchema(() => z.object({ - /** - * Conflict resolution strategy - */ - conflictResolution: z.enum(['skip', 'overwrite', 'merge', 'fail']) - .default('merge') - .describe('How to handle existing items'), - - /** - * Validate items against schema - */ - validate: z.boolean().default(true).describe('Validate before import'), - - /** - * Dry run (don't actually save) - */ - dryRun: z.boolean().default(false).describe('Simulate import without saving'), - - /** - * Continue on errors - */ - continueOnError: z.boolean().default(false).describe('Continue if validation fails'), - - /** - * Transform function (as string) - * Example: "(item) => ({ ...item, imported: true })" - */ - transform: z.string().optional().describe('Transform items before import'), -})); +// Until #4411 this file ALSO declared its own copy of all eleven of those +// envelope types. Each name existed twice across two subpath entries +// (`@objectstack/spec/kernel` and `@objectstack/spec/system`) with a different +// shape, so which one you got depended on your import path — a coin-flip an +// auto-import or a model completion has no way to win on purpose, and the +// stricter-looking, more heavily documented copy was the DEAD one. Every +// consumer in this repo, `cloud` and `objectui` imported the `system` copy; +// the kernel copies had zero runtime consumers and only their own test +// parsing them, so they were removed under ADR-0049 enforce-or-remove. +// Manager *wiring* stays here; the *envelope* is owned by `system`. -/** - * Metadata Loader Result - * Result of a metadata load operation - */ -export const MetadataLoadResultSchema = lazySchema(() => z.object({ - /** - * Loaded data - */ - data: z.unknown().nullable().describe('Loaded metadata'), - - /** - * Whether data came from cache (304 Not Modified) - */ - fromCache: z.boolean().default(false).describe('Loaded from cache'), - - /** - * Not modified (conditional request matched) - */ - notModified: z.boolean().default(false).describe('Not modified since last request'), - - /** - * ETag of loaded data - */ - etag: z.string().optional().describe('Entity tag'), - - /** - * Statistics about loaded data - */ - stats: MetadataStatsSchema.optional().describe('Metadata statistics'), - - /** - * Load time in milliseconds - */ - loadTime: z.number().min(0).optional().describe('Load duration in ms'), -})); - -/** - * Metadata Save Result - */ -export const MetadataSaveResultSchema = lazySchema(() => z.object({ - /** - * Whether save was successful - */ - success: z.boolean().describe('Save successful'), - - /** - * Path where file was saved - */ - path: z.string().describe('Output path'), - - /** - * Generated ETag - */ - etag: z.string().optional().describe('Generated entity tag'), - - /** - * File size in bytes - */ - size: z.number().int().min(0).optional().describe('File size'), - - /** - * Save time in milliseconds - */ - saveTime: z.number().min(0).optional().describe('Save duration in ms'), - - /** - * Backup path (if created) - */ - backupPath: z.string().optional().describe('Backup file path'), -})); - -/** - * Metadata Watch Event - */ -export const MetadataWatchEventSchema = lazySchema(() => z.object({ - /** - * Event type - */ - type: z.enum(['added', 'changed', 'deleted']).describe('Event type'), - - /** - * Metadata type (e.g., 'object', 'view', 'app') - */ - metadataType: z.string().describe('Type of metadata'), - - /** - * Item name/identifier - */ - name: z.string().describe('Item identifier'), - - /** - * Full file path - */ - path: z.string().describe('File path'), - - /** - * Loaded item data (for added/changed events) - */ - data: z.unknown().optional().describe('Item data'), - - /** - * Timestamp - */ - timestamp: z.string().datetime().describe('Event timestamp'), -})); - -/** - * Metadata Collection Info - * Summary of a metadata collection - */ -export const MetadataCollectionInfoSchema = lazySchema(() => z.object({ - /** - * Collection type (e.g., 'object', 'view', 'app') - */ - type: z.string().describe('Collection type'), - - /** - * Total items in collection - */ - count: z.number().int().min(0).describe('Number of items'), - - /** - * Formats found in collection - */ - formats: z.array(MetadataFormatSchema).describe('Formats in collection'), - - /** - * Total size in bytes - */ - totalSize: z.number().int().min(0).optional().describe('Total size in bytes'), - - /** - * Last modified timestamp - */ - lastModified: z.string().datetime().optional().describe('Last modification date'), - - /** - * Collection location (path or URL) - */ - location: z.string().optional().describe('Collection location'), -})); - -/** - * Metadata Loader Interface Contract - * Defines the standard methods all metadata loaders must implement - */ -export const MetadataLoaderContractSchema = lazySchema(() => z.object({ - /** - * Loader name/identifier - */ - name: z.string().describe('Loader identifier'), - - /** - * Protocol handled by this loader (e.g. 'file:', 'http:', 's3:', 'datasource:') - */ - protocol: z.enum(['file:', 'http:', 's3:', 'datasource:', 'memory:']).describe('Protocol identifier'), - - /** - * Detailed capabilities - */ - capabilities: z.object({ - read: z.boolean().default(true), - write: z.boolean().default(false), - watch: z.boolean().default(false), - list: z.boolean().default(true), - }).describe('Loader capabilities'), - - /** - * Supported formats - */ - supportedFormats: z.array(MetadataFormatSchema).describe('Supported formats'), - - /** - * Whether loader supports watching for changes - */ - supportsWatch: z.boolean().default(false).describe('Supports file watching'), - - /** - * Whether loader supports saving - */ - supportsWrite: z.boolean().default(true).describe('Supports write operations'), - - /** - * Whether loader supports caching - */ - supportsCache: z.boolean().default(true).describe('Supports caching'), -})); +import { lazySchema } from '../shared/lazy-schema'; +// `MetadataManagerConfig.formats` is the ONLY surviving use of a format enum in +// this file. It reads the `shared` copy rather than declaring a fourth one — +// same four members, and `shared` is a leaf module so there is no cycle back +// through `system`. Deliberately NOT the `system` enum: that one is a wider +// superset (`yml`/`ts`/`js` aliases) and adopting it here would silently widen +// what this config accepts. +import { MetadataFormatSchema } from '../shared/metadata-types.zod'; /** * Metadata Fallback Strategy @@ -446,12 +67,12 @@ export const MetadataManagerConfigSchema = lazySchema(() => z.object({ * Root directory for metadata (for filesystem loaders) */ rootDir: z.string().optional().describe('Root directory path'), - + /** * Enabled serialization formats */ formats: z.array(MetadataFormatSchema).default(['typescript', 'json', 'yaml']).describe('Enabled formats'), - + /** * Cache configuration */ @@ -474,12 +95,12 @@ export const MetadataManagerConfigSchema = lazySchema(() => z.object({ ttl: z.number().int().min(0).default(60_000).describe('Cache TTL in milliseconds'), }).optional().describe('DatabaseLoader read-through cache'), }).optional().describe('Cache settings'), - + /** * Watch for file changes */ watch: z.boolean().default(false).describe('Enable file watching'), - + /** * Watch options */ @@ -488,7 +109,7 @@ export const MetadataManagerConfigSchema = lazySchema(() => z.object({ persistent: z.boolean().default(true).describe('Keep process running'), ignoreInitial: z.boolean().default(true).describe('Ignore initial add events'), }).optional().describe('File watcher options'), - + /** * Validation settings */ @@ -496,7 +117,7 @@ export const MetadataManagerConfigSchema = lazySchema(() => z.object({ strict: z.boolean().default(true).describe('Strict validation'), throwOnError: z.boolean().default(true).describe('Throw on validation error'), }).optional().describe('Validation settings'), - + /** * Loader-specific options */ @@ -525,16 +146,5 @@ export const MetadataManagerConfigSchema = lazySchema(() => z.object({ })); // Export types -export type MetadataFormat = z.infer; -export type MetadataStats = z.infer; -export type MetadataLoadOptions = z.input; -export type MetadataSaveOptions = z.infer; -export type MetadataExportOptions = z.infer; -export type MetadataImportOptions = z.infer; -export type MetadataLoadResult = z.infer; -export type MetadataSaveResult = z.infer; -export type MetadataWatchEvent = z.infer; -export type MetadataCollectionInfo = z.infer; -export type MetadataLoaderContract = z.input; export type MetadataManagerConfig = z.input; export type MetadataFallbackStrategy = z.infer; diff --git a/packages/spec/src/kernel/metadata-plugin.zod.ts b/packages/spec/src/kernel/metadata-plugin.zod.ts index cce0f3c65f..547424f434 100644 --- a/packages/spec/src/kernel/metadata-plugin.zod.ts +++ b/packages/spec/src/kernel/metadata-plugin.zod.ts @@ -37,9 +37,9 @@ import { ActionSchema } from '../ui/action.zod'; * - **Kubernetes**: API Server + CRD Registry * * ## References - * - kernel/metadata-loader.zod.ts — Storage backend protocol + * - kernel/metadata-loader.zod.ts — MetadataManager wiring (datasource, cache, write gates) * - kernel/metadata-customization.zod.ts — Overlay/merge protocol - * - system/metadata-persistence.zod.ts — Database record format + * - system/metadata-persistence.zod.ts — Database record format + loader/watch envelope types * - contracts/metadata-service.ts — Service interface */ diff --git a/packages/spec/src/system/metadata-persistence.zod.ts b/packages/spec/src/system/metadata-persistence.zod.ts index ed0e917a63..b94c8fc6c1 100644 --- a/packages/spec/src/system/metadata-persistence.zod.ts +++ b/packages/spec/src/system/metadata-persistence.zod.ts @@ -161,6 +161,15 @@ export const PackagePublishResultSchema = lazySchema(() => z.object({ export type PackagePublishResult = z.infer; +// ─── Loader / watch envelope types ─────────────────────────────────────────── +// +// Everything from here to `MetadataSource` below is the SINGLE source for the +// metadata loader + watch vocabulary. `kernel/metadata-loader.zod` used to +// declare a differently-shaped copy of each of these names on the +// `@objectstack/spec/kernel` entry — an import-path coin-flip that no consumer +// ever won on purpose (every one of them imported from here). The kernel copies +// had zero consumers and were removed in #4411; keep new envelope types here. + /** * Metadata Format * Supported file formats for metadata serialization. @@ -322,6 +331,10 @@ export const MetadataSourceSchema = lazySchema(() => z.enum([ * historically declared a narrower duplicate; we re-export the kernel version * here so a single TypeScript type is observed everywhere `@objectstack/spec` * consumers reach for it. + * + * This pair is the ONLY thing this file takes from kernel, and it is the + * direction that survived #4411: manager *wiring* is owned by kernel, the + * loader/watch *envelope* is owned here. Nothing is declared twice. */ export { MetadataFallbackStrategySchema, diff --git a/skills/objectstack-platform/references/_index.md b/skills/objectstack-platform/references/_index.md index bd86e85676..e8494fb400 100644 --- a/skills/objectstack-platform/references/_index.md +++ b/skills/objectstack-platform/references/_index.md @@ -26,10 +26,11 @@ from `node_modules` — there is no local copy in the skill bundle. - `node_modules/@objectstack/spec/src/data/hook-body.zod.ts` — Capability tokens a script body may request. - `node_modules/@objectstack/spec/src/kernel/cluster.zod.ts` — Cluster Protocol - `node_modules/@objectstack/spec/src/kernel/metadata-customization.zod.ts` — Metadata Customization Layer Protocol -- `node_modules/@objectstack/spec/src/kernel/metadata-loader.zod.ts` — Metadata Loader Protocol +- `node_modules/@objectstack/spec/src/kernel/metadata-loader.zod.ts` — Metadata Manager Configuration - `node_modules/@objectstack/spec/src/kernel/metadata-protection.zod.ts` — Metadata Protection Model — Phase 1 (ADR-0010) - `node_modules/@objectstack/spec/src/shared/expression.zod.ts` — Expression Protocol - `node_modules/@objectstack/spec/src/shared/identifiers.zod.ts` — System Identifier Schema +- `node_modules/@objectstack/spec/src/shared/metadata-types.zod.ts` — Exports: MetadataFormatSchema, BaseMetadataRecordSchema - `node_modules/@objectstack/spec/src/shared/protection.zod.ts` — Package-level metadata protection (ADR-0010 §3.7 — Phase 4.3) - `node_modules/@objectstack/spec/src/shared/suggestions.zod.ts` — "Did you mean?" Suggestion Utilities - `node_modules/@objectstack/spec/src/system/tenant.zod.ts` — Tenant Schema (Multi-Tenant Architecture) From e3f2ad6712a35fdab7a80438be82508ca12df026 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 1 Aug 2026 09:24:51 +0000 Subject: [PATCH 2/2] chore(spec): write the hand-edited baselines the way the generator does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two baselines this branch edits by hand — `json-schema.manifest.json` and `authorable-surface.json` — came out with `—` escaped as `—`, because the edit went through Python's `json.dump`, whose `ensure_ascii` defaults to true. `build-schemas.ts` writes them with `JSON.stringify`, which emits the character literally. No gate catches this: the manifest is only rewritten when the SCHEMA KEY SET changes, so the escape would have sat in the file until the next PR that adds a schema, where the generator would silently rewrite it back and hand that author an unrelated one-line diff to explain. Re-serialised with `ensure_ascii=False`. Both files now differ from main by exactly the intended removals and nothing else: 11 manifest keys, 65 authorable-surface lines, zero incidental churn. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01M9uWvoEp9CoLzYjNExj9sL --- packages/spec/authorable-surface.json | 2 +- packages/spec/json-schema.manifest.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/spec/authorable-surface.json b/packages/spec/authorable-surface.json index bec27c18e7..5cf1f44283 100644 --- a/packages/spec/authorable-surface.json +++ b/packages/spec/authorable-surface.json @@ -1,5 +1,5 @@ { - "description": "Ratchet of every AUTHORABLE key in the spec \u2014 what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 \u00a75.", + "description": "Ratchet of every AUTHORABLE key in the spec — what a metadata author may write, which for this platform IS the third-party API. Auto-updated on additions (commit the change). A key that disappears without a tombstone fails gen:schema, because these schemas are not .strict() and Zod would silently strip it. \"[RETIRED]\" marks a tombstoned key that still rejects with an upgrade prescription. See #3855, ADR-0059 §5.", "keys": [ "ai/AIModelConfig:maxTokens", "ai/AIModelConfig:model", diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index 271ebdc1a7..85eeb4c1e0 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -1,5 +1,5 @@ { - "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema \u2014 remove a key ONLY for a deliberate retirement. See #2978.", + "description": "Ratchet manifest of every JSON Schema emitted by scripts/build-schemas.ts. Auto-appended when new schemas are added (commit the change). A listed schema that a build no longer emits fails gen:schema — remove a key ONLY for a deliberate retirement. See #2978.", "schemas": [ "ai/AIModelConfig", "ai/AIUsageRecord",