From 86ce51dfd2f67d9fce6371b752a0f98ae8e5e3c6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 17:54:42 +0000 Subject: [PATCH 1/3] refactor(spec)!: retire external-lookup and message-queue config families (ADR-0049, #8075) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fork (b) of the #8075 census: both modules are dead declarations whose only distinctive feature was an inline-credential sink. Route 3 retirement — no tombstone, no D2 conversion; RETIRED_DEFS_BY_MAJOR[17] (8 defs) + the D3 SemanticMigration external-lookup-message-queue-families-retired are the declaration. The #5552 data/ExternalFieldMapping:transform retired-keys entry is subsumed by the def retirement. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0123k4cam2jEAkPmbJeoaY3r --- ...l-lookup-message-queue-families-retired.md | 58 ++ .../docs/getting-started/quick-reference.mdx | 6 +- .../docs/references/data/external-lookup.mdx | 97 --- content/docs/references/data/index.mdx | 1 - content/docs/references/data/meta.json | 1 - content/docs/references/index.mdx | 14 +- content/docs/references/shared/mapping.mdx | 6 +- content/docs/references/system/index.mdx | 1 - .../docs/references/system/message-queue.mdx | 108 --- content/docs/references/system/meta.json | 1 - docs/protocol-upgrade-guide.md | 3 + packages/spec/api-surface/data.json | 8 - packages/spec/api-surface/system.json | 14 - packages/spec/authorable-defaults/data.json | 1 - packages/spec/authorable-defaults/system.json | 9 - packages/spec/authorable-surface/data.json | 21 - packages/spec/authorable-surface/system.json | 18 - packages/spec/export-origins/data.json | 8 - packages/spec/export-origins/system.json | 14 - packages/spec/json-schema.manifest/data.json | 3 - .../spec/json-schema.manifest/system.json | 5 - packages/spec/scripts/build-docs.ts | 8 +- packages/spec/scripts/lib/renamed-defs.ts | 3 +- packages/spec/scripts/renamed-defs.test.ts | 8 +- packages/spec/spec-changes.json | 14 + packages/spec/src/conversions/registry.ts | 9 +- .../data/external-lookup-retirement.test.ts | 161 ++++ .../spec/src/data/external-lookup.test.ts | 817 ------------------ packages/spec/src/data/external-lookup.zod.ts | 315 ------- packages/spec/src/data/index.ts | 23 +- .../spec/src/integration/connector.test.ts | 13 +- .../spec/src/integration/connector.zod.ts | 7 +- .../17.data__ExternalDataSource.ts | 14 + .../17.data__ExternalFieldMapping.ts | 12 + .../retired-defs/17.data__ExternalLookup.ts | 11 + .../retired-defs/17.system__ConsumerConfig.ts | 7 + .../17.system__DeadLetterQueue.ts | 9 + .../17.system__MessageQueueConfig.ts | 14 + .../17.system__MessageQueueProvider.ts | 9 + .../retired-defs/17.system__TopicConfig.ts | 8 + ...7.data__ExternalFieldMapping__transform.ts | 3 - ...l-lookup-message-queue-families-retired.ts | 73 ++ packages/spec/src/migrations/registry.ts | 151 +++- packages/spec/src/shared/mapping.test.ts | 15 +- packages/spec/src/shared/mapping.zod.ts | 6 +- packages/spec/src/system/index.ts | 17 +- .../system/message-queue-retirement.test.ts | 169 ++++ .../spec/src/system/message-queue.test.ts | 202 ----- packages/spec/src/system/message-queue.zod.ts | 69 -- .../src/type-alias-convention.pin.test.ts | 8 +- 50 files changed, 801 insertions(+), 1771 deletions(-) create mode 100644 .changeset/external-lookup-message-queue-families-retired.md delete mode 100644 content/docs/references/data/external-lookup.mdx delete mode 100644 content/docs/references/system/message-queue.mdx create mode 100644 packages/spec/src/data/external-lookup-retirement.test.ts delete mode 100644 packages/spec/src/data/external-lookup.test.ts delete mode 100644 packages/spec/src/data/external-lookup.zod.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.data__ExternalDataSource.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.data__ExternalFieldMapping.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.data__ExternalLookup.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.system__ConsumerConfig.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.system__DeadLetterQueue.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueConfig.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueProvider.ts create mode 100644 packages/spec/src/migrations/entries/retired-defs/17.system__TopicConfig.ts delete mode 100644 packages/spec/src/migrations/entries/retired-keys/17.data__ExternalFieldMapping__transform.ts create mode 100644 packages/spec/src/migrations/entries/semantic/17.external-lookup-message-queue-families-retired.ts create mode 100644 packages/spec/src/system/message-queue-retirement.test.ts delete mode 100644 packages/spec/src/system/message-queue.test.ts delete mode 100644 packages/spec/src/system/message-queue.zod.ts diff --git a/.changeset/external-lookup-message-queue-families-retired.md b/.changeset/external-lookup-message-queue-families-retired.md new file mode 100644 index 0000000000..667a1250c7 --- /dev/null +++ b/.changeset/external-lookup-message-queue-families-retired.md @@ -0,0 +1,58 @@ +--- +"@objectstack/spec": major +--- + +refactor(spec)!: retire the external-lookup and message-queue config families — two dead declarations whose only distinctive feature was an inline-credential sink (#8075) + +`ExternalDataSource(Schema)`, `ExternalFieldMapping(Schema|Parsed)`, +`ExternalLookup(Schema|Parsed)` (the whole of `data/external-lookup.zod.ts`) and +`MessageQueueConfig(Schema|Parsed)`, `MessageQueueProvider(Schema)`, +`TopicConfig(Schema|Parsed)`, `ConsumerConfig(Schema|Parsed)`, +`DeadLetterQueue(Schema|Parsed)` (the whole of `system/message-queue.zod.ts`) are +REMOVED under ADR-0049 enforce-or-remove — 8 defs, 22 exported names, reference docs +with them. + +Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): +security-shaped declared surface with inline-credential sinks and **zero consumers**. +`ExternalDataSourceSchema.authentication.config` was a record whose own docblock +example wrote `"clientSecret": "..."` inline; `MessageQueueConfigSchema.sasl` required +an inline `password` whenever present. Neither schema was reachable from any +metadata-type binding, stack collection or `/meta` door, and neither had a single +import outside `packages/spec` repo-wide (corpus-reach control passing in the same +run). The consumed near-namesake `kernel/EventMessageQueueConfig` deliberately carries +NO credential field — so the consumed MQ shape had no credential key and the +credential-bearing MQ shape had no consumer. A dead schema minus one field is still a +dead schema, so the whole declarations go, not just the credential faces (#3950). + +The #7990 Option-B reopen trigger ("a third measured artefact-type surface") is NOT +met — nothing ever persisted these; this is the ADR-0049 leg of the triage-agreed +fork. + +FROM → TO: + +| removed | use instead | +|---|---| +| `ExternalLookup` / `ExternalDataSource` (+ `authentication.config` inline secrets) | `object.external` (`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference; connection credentials live in datasource config (`data/datasource.zod.ts`, `data/driver/`), never inline in object metadata. `data/external-catalog.zod.ts` is that path's catalog surface and is untouched | +| `ExternalFieldMapping` | **nothing** — it existed only to serve `ExternalLookup.fieldMappings`. The base `shared/FieldMapping` and `integration/ConnectorFieldMapping` are untouched | +| `MessageQueueConfig` (+ `sasl.username`/`sasl.password`) / `MessageQueueProvider` / `TopicConfig` / `ConsumerConfig` / `DeadLetterQueue` | the live MQ surface is `kernel/EventMessageQueueConfig` (`EventBusConfig.messageQueue`) — topic, pattern, format, batching; **no credential field by design**. Broker connection + SASL credentials are runtime deployment configuration, not authorable metadata. `kernel/DeadLetterQueueEntry` (the event bus's per-event DLQ record) is untouched | + +**The fix:** delete the import. Nothing was ever deployed under either family — that +is the finding, not a consolation — so there is no data migration; `tsc` reports +TS2305 at every import of a retired name. Either capability returns via the ENFORCE +route of ADR-0049 through a new ADR: the executor / broker admin service first, the +vocabulary second. + +**Subsumed:** the #5552 `data/ExternalFieldMapping:transform` `retiredKey()` tombstone +and its `RETIRED_KEYS_BY_MAJOR[17]` entry — both land in the unreleased protocol 17, +so composed, the key retirement is absorbed by the def retirement (the +`WidgetManifest.performance` way): there is no longer a mapping shape to author the +key INTO. The `shared/FieldMapping` tombstone and the +`integration/ConnectorFieldMapping` spelling still reject `transform` with the #5552 +prescription; the `field-mapping-transform-removed` D2 conversion still rewrites +`connectors[].fieldMappings[].transform`. + +The retirement kit — route 3: no tombstone, no D2 conversion. +`RETIRED_DEFS_BY_MAJOR[17]` (8 defs) plus the D3 `SemanticMigration` +`external-lookup-message-queue-families-retired` are the declaration. + + diff --git a/content/docs/getting-started/quick-reference.mdx b/content/docs/getting-started/quick-reference.mdx index 0989ba0fea..1c4d5475fe 100644 --- a/content/docs/getting-started/quick-reference.mdx +++ b/content/docs/getting-started/quick-reference.mdx @@ -23,7 +23,7 @@ Categories that have no section here at all are named under [Categories Without a Section](#categories-without-a-section) — that curation is stated, not left implicit. -## Data Protocol (17 of 30 schemas) +## Data Protocol (16 of 29 schemas) Core business logic and data modeling schemas. @@ -43,7 +43,6 @@ Core business logic and data modeling schemas. | **[SQL Driver](/docs/references/data/driver-sql)** | `driver-sql.zod.ts` | SQLDriverConfig, SQLDialect | SQL-specific driver | | **[NoSQL Driver](/docs/references/data/driver-nosql)** | `driver-nosql.zod.ts` | NoSQLDriverConfig | NoSQL-specific driver | | **[Document](/docs/references/data/document)** | `document.zod.ts` | Document | Document-oriented data | -| **[External Lookup](/docs/references/data/external-lookup)** | `external-lookup.zod.ts` | ExternalLookup | External data lookups | | **[Postgres Driver](/docs/references/data/driver-postgres)** | `driver/postgres.zod.ts` | PostgresConfig | PostgreSQL configuration | | **[Mongo Driver](/docs/references/data/driver-mongo)** | `driver/mongo.zod.ts` | MongoConfig | MongoDB configuration | @@ -89,7 +88,7 @@ Plugin architecture, manifests, and kernel runtime. | **[Metadata Loader](/docs/references/kernel/metadata-loader)** | `metadata-loader.zod.ts` | MetadataLoaderContract | Metadata loading | | **[Package Registry](/docs/references/kernel/package-registry)** | `package-registry.zod.ts` | InstalledPackage, InstallPackageRequest | Package resolution | -## System Protocol (18 of 37 schemas) +## System Protocol (17 of 36 schemas) Runtime environment, logging, jobs, caching, and observability. @@ -103,7 +102,6 @@ Runtime environment, logging, jobs, caching, and observability. | **[HTTP Server](/docs/references/system/http-server)** | `http-server.zod.ts` | RouteHandlerMetadata, MiddlewareConfig | Route + middleware metadata | | **[Job](/docs/references/system/job)** | `job.zod.ts` | Job, JobSchedule | Background job queue | | **[Logging](/docs/references/system/logging)** | `logging.zod.ts` | LoggingConfig | Structured logging | -| **[Message Queue](/docs/references/system/message-queue)** | `message-queue.zod.ts` | MessageQueueConfig, TopicConfig | Message queuing | | **[Metadata Persistence](/docs/references/system/metadata-persistence)** | `metadata-persistence.zod.ts` | MetadataHistoryRecord, MetadataDiffResult | Metadata storage | | **[Metrics](/docs/references/system/metrics)** | `metrics.zod.ts` | Metrics | Application metrics | | **[Migration](/docs/references/system/migration)** | `migration.zod.ts` | Migration | Schema migration | diff --git a/content/docs/references/data/external-lookup.mdx b/content/docs/references/data/external-lookup.mdx deleted file mode 100644 index bb77e2ead6..0000000000 --- a/content/docs/references/data/external-lookup.mdx +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: External Lookup -description: External Lookup protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -External Data Source Schema - -Configuration for connecting to external data systems. -Similar to Salesforce External Objects for real-time data integration. - -@example -```json -{ - "id": "salesforce-accounts", - "name": "Salesforce Account Data", - "type": "rest-api", - "endpoint": "https://api.salesforce.com/services/data/v58.0", - "authentication": { - "type": "oauth2", - "config": { - "clientId": "...", - "clientSecret": "...", - "tokenUrl": "https://login.salesforce.com/services/oauth2/token" - } - } -} -``` - - -**Source:** `packages/spec/src/data/external-lookup.zod.ts` - - -## TypeScript Usage - -```typescript -import { ExternalDataSourceSchema, ExternalFieldMappingSchema, ExternalLookupSchema } from '@objectstack/spec/data'; -import type { ExternalDataSource, ExternalFieldMapping, ExternalLookup } from '@objectstack/spec/data'; - -// Validate data -const result = ExternalDataSourceSchema.parse(data); -``` - ---- - -## ExternalDataSource - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **id** | `string` | ✅ | Data source ID | -| **name** | `string` | ✅ | Data source name | -| **type** | `Enum<'odata' \| 'rest-api' \| 'graphql' \| 'custom'>` | ✅ | Protocol type | -| **endpoint** | `string` | ✅ | API endpoint URL | -| **authentication** | `{ type: Enum<'oauth2' \| 'api-key' \| 'basic' \| 'none'>; config: Record }` | ✅ | Authentication | - - ---- - -## ExternalFieldMapping - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **source** | `string` | ✅ | Source field name | -| **target** | `string` | ✅ | Target field name | -| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite existing sources automatically. | -| **defaultValue** | `any` | optional | Default if source is null/undefined | -| **type** | `string` | optional | Field type | -| **readonly** | `boolean` | ✅ | Read-only field | - - ---- - -## ExternalLookup - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **fieldName** | `string` | ✅ | Field name | -| **dataSource** | `{ id: string; name: string; type: Enum<'odata' \| 'rest-api' \| 'graphql' \| 'custom'>; endpoint: string; … }` | ✅ | External data source | -| **query** | `{ endpoint: string; method: Enum<'GET' \| 'POST'>; parameters?: Record }` | ✅ | Query configuration | -| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; type?: string; … }[]` | ✅ | Field mappings | -| **caching** | `{ enabled: boolean; ttl: number; strategy: Enum<'lru' \| 'lfu' \| 'ttl'> }` | optional | Caching configuration | -| **fallback** | `{ enabled: boolean; defaultValue?: any; showError: boolean }` | optional | Fallback configuration | -| **rateLimit** | `{ requestsPerSecond: number; burstSize?: number }` | optional | Rate limiting | -| **retry** | `{ maxRetries: number; initialDelayMs: number; maxDelayMs: number; backoffMultiplier: number; … }` | optional | Retry configuration with exponential backoff | -| **transform** | `{ request?: object; response?: object }` | optional | Request/response transformation pipeline | -| **pagination** | `{ type: Enum<'offset' \| 'cursor' \| 'page'>; pageSize: number; maxPages?: number }` | optional | Pagination configuration for external data | - - ---- - diff --git a/content/docs/references/data/index.mdx b/content/docs/references/data/index.mdx index f98ecea006..ad98cb1db4 100644 --- a/content/docs/references/data/index.mdx +++ b/content/docs/references/data/index.mdx @@ -23,7 +23,6 @@ This section contains all protocol schemas for the data layer of ObjectStack. - diff --git a/content/docs/references/data/meta.json b/content/docs/references/data/meta.json index ab512bcd57..ec5a53503d 100644 --- a/content/docs/references/data/meta.json +++ b/content/docs/references/data/meta.json @@ -20,7 +20,6 @@ "driver-nosql", "driver-sql", "external-catalog", - "external-lookup", "---Documents & Seed---", "document", "feed", diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 252fee6447..e554ac9100 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,6 +1,6 @@ --- title: Protocol Reference -description: Every schema published by @objectstack/spec — 1578 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1570 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -23,7 +23,7 @@ counts are sums of the rows they head. Regenerate with | [API Protocol](/docs/references/api) | 28 | 412 | REST/GraphQL contracts, endpoints, routing, realtime, batch, discovery. | | [Automation Protocol](/docs/references/automation) | 13 | 68 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. | | [Cloud Protocol](/docs/references/cloud) | 11 | 94 | Environments, packages and versions, marketplace, developer portal, tenancy. | -| [Data Protocol](/docs/references/data) | 30 | 166 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | +| [Data Protocol](/docs/references/data) | 29 | 163 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | | [Identity Protocol](/docs/references/identity) | 5 | 28 | Users and accounts, organizations, positions, API keys, SCIM provisioning. | | [Integration Protocol](/docs/references/integration) | 1 | 27 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. | | [Kernel Protocol](/docs/references/kernel) | 31 | 176 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. | @@ -31,9 +31,9 @@ counts are sums of the rows they head. Regenerate with | [Security Protocol](/docs/references/security) | 5 | 27 | Permission sets, row-level security, sharing rules, tenancy posture. | | [Shared Protocol](/docs/references/shared) | 8 | 31 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. | | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | -| [System Protocol](/docs/references/system) | 37 | 292 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | +| [System Protocol](/docs/references/system) | 36 | 287 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 148 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **200** | **1578** | 14 protocol modules | +| **Total** | **198** | **1570** | 14 protocol modules | --- @@ -146,7 +146,7 @@ Environments, packages and versions, marketplace, developer portal, tenancy. ## Data Protocol -**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **30 pages, 166 schemas** +**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 163 schemas** Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. @@ -169,7 +169,6 @@ Objects, fields, queries, filters, datasources and drivers — the ObjectQL laye | [`driver/sqlite.zod.ts`](/docs/references/data/driver-sqlite) | `SqliteConfig`, `SqliteWasmConfig`, `SqliteWasmPersistMode` | | [`driver/turso.zod.ts`](/docs/references/data/driver-turso) | `TursoConfig`, `TursoTransportMode` | | [`external-catalog.zod.ts`](/docs/references/data/external-catalog) | `ExternalCatalog`, `ExternalColumn`, `ExternalTable` | -| [`external-lookup.zod.ts`](/docs/references/data/external-lookup) | `ExternalDataSource`, `ExternalFieldMapping`, `ExternalLookup` | | [`feed.zod.ts`](/docs/references/data/feed) | `FeedFilterMode`, `FeedItemType` | | [`field.zod.ts`](/docs/references/data/field) | `CurrencyConfig`, `CurrencyValue`, `Field`, `FieldType`, `LocationCoordinates`, `SelectOption`, `UniqueScope` | | [`field-value.zod.ts`](/docs/references/data/field-value) | `Address`, `AddressValue`, `CalendarDateValue`, `ClockTimeValue`, `FileLikeValue`, `FileReferenceIdValue`, `FileValue`, `InstantValue`, `LocationValue`, `ReferenceIdValue` | @@ -318,7 +317,7 @@ Studio designer metadata — the authoring surfaces for the protocols above. ## System Protocol -**Source:** `packages/spec/src/system/` · **Import:** `@objectstack/spec/system` · **37 pages, 292 schemas** +**Source:** `packages/spec/src/system/` · **Import:** `@objectstack/spec/system` · **36 pages, 287 schemas** The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. @@ -343,7 +342,6 @@ The runtime environment — logging, jobs, cache, metrics, notifications, i18n a | [`job.zod.ts`](/docs/references/system/job) | `CronSchedule`, `IntervalSchedule`, `Job`, `JobExecution`, `JobExecutionStatus`, `OnceSchedule`, `RetryPolicy`, `Schedule` | | [`license.zod.ts`](/docs/references/system/license) | `Feature`, `License`, `LicenseMetricType`, `Plan` | | [`logging.zod.ts`](/docs/references/system/logging) | `ConsoleDestinationConfig`, `ExtendedLogLevel`, `ExternalServiceDestinationConfig`, `FileDestinationConfig`, `HttpDestinationConfig`, `LogDestination`, `LogDestinationType`, `LogEnrichmentConfig`, `LogEntry`, `LogFormat`, `LogLevel`, `LoggerConfig`, `LoggingConfig`, `StructuredLogEntry` | -| [`message-queue.zod.ts`](/docs/references/system/message-queue) | `ConsumerConfig`, `DeadLetterQueue`, `MessageQueueConfig`, `MessageQueueProvider`, `TopicConfig` | | [`metadata-persistence.zod.ts`](/docs/references/system/metadata-persistence) | `MetadataCollectionInfo`, `MetadataDiffResult`, `MetadataFallbackStrategy`, `MetadataFormat`, `MetadataHistoryQueryOptions`, `MetadataHistoryQueryResult`, `MetadataHistoryRecord`, `MetadataHistoryRetentionPolicy`, `MetadataLoadOptions`, `MetadataLoadResult`, `MetadataLoaderContract`, `MetadataManagerConfig`, `MetadataRecord`, `MetadataSaveOptions`, `MetadataSaveResult`, `MetadataScope`, `MetadataSource`, `MetadataState`, `MetadataStats`, `MetadataWatchEvent`, `PackagePublishResult` | | [`metrics.zod.ts`](/docs/references/system/metrics) | `HistogramBucketConfig`, `MetricAggregationConfig`, `MetricAggregationType`, `MetricDataPoint`, `MetricDefinition`, `MetricExportConfig`, `MetricLabels`, `MetricType`, `MetricUnit`, `MetricsConfig`, `ServiceLevelIndicator`, `ServiceLevelObjective`, `TimeSeries`, `TimeSeriesDataPoint` | | [`migration.zod.ts`](/docs/references/system/migration) | `AddFieldOperation`, `ChangeSet`, `CreateObjectOperation`, `DataMigrationFlag`, `DeleteObjectOperation`, `ExecuteSqlOperation`, `MigrationDependency`, `MigrationJournalEvent`, `MigrationOperation`, `ModifyFieldOperation`, `RemoveFieldOperation`, `RenameObjectOperation` | diff --git a/content/docs/references/shared/mapping.mdx b/content/docs/references/shared/mapping.mdx index 6b0e718ee4..abb981e314 100644 --- a/content/docs/references/shared/mapping.mdx +++ b/content/docs/references/shared/mapping.mdx @@ -7,14 +7,16 @@ description: Mapping protocol schemas Base Field Mapping Protocol -Shared by: Connector, External Lookup +Shared by: Connector This module provides the canonical field mapping schema used across ObjectStack for data synchronization. **Use Cases:** - Integration connectors ([integration/connector.zod.ts](/docs/references/integration/connector)) -- External lookups ([data/external-lookup.zod.ts](/docs/references/data/external-lookup)) +- (`data/external-lookup.zod.ts`'s `ExternalFieldMappingSchema` extended + this base too, until the whole external-lookup family was retired in + #8075 — ADR-0049, zero consumers.) @example Basic field mapping ```typescript diff --git a/content/docs/references/system/index.mdx b/content/docs/references/system/index.mdx index 1b7d6c1222..f2b84b82d3 100644 --- a/content/docs/references/system/index.mdx +++ b/content/docs/references/system/index.mdx @@ -25,7 +25,6 @@ This section contains all protocol schemas for the system layer of ObjectStack. - diff --git a/content/docs/references/system/message-queue.mdx b/content/docs/references/system/message-queue.mdx deleted file mode 100644 index c43c627abb..0000000000 --- a/content/docs/references/system/message-queue.mdx +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: Message Queue -description: Message Queue protocol schemas ---- - -{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} - -Message queue protocol for async communication -Supports Kafka, RabbitMQ, AWS SQS, Redis Pub/Sub - - -**Source:** `packages/spec/src/system/message-queue.zod.ts` - - -## TypeScript Usage - -```typescript -import { ConsumerConfigSchema, DeadLetterQueueSchema, MessageQueueConfigSchema, MessageQueueProviderSchema, TopicConfigSchema } from '@objectstack/spec/system'; -import type { ConsumerConfig, DeadLetterQueue, MessageQueueConfig, MessageQueueProvider, TopicConfig } from '@objectstack/spec/system'; - -// Validate data -const result = ConsumerConfigSchema.parse(data); -``` - ---- - -## ConsumerConfig - -Consumer group configuration for topic consumption - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **groupId** | `string` | ✅ | Consumer group identifier | -| **autoOffsetReset** | `Enum<'earliest' \| 'latest'>` | ✅ | Where to start reading when no offset exists | -| **enableAutoCommit** | `boolean` | ✅ | Automatically commit consumed offsets | -| **maxPollRecords** | `number` | ✅ | Maximum records returned per poll | - - ---- - -## DeadLetterQueue - -Dead letter queue configuration for unprocessable messages - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **enabled** | `boolean` | ✅ | Enable dead letter queue for failed messages | -| **maxRetries** | `number` | ✅ | Maximum delivery attempts before sending to DLQ | -| **queueName** | `string` | ✅ | Name of the dead letter queue | - - ---- - -## MessageQueueConfig - -Top-level message queue configuration - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **provider** | `Enum<'kafka' \| 'rabbitmq' \| 'aws-sqs' \| 'redis-pubsub' \| 'google-pubsub' \| 'azure-service-bus'>` | ✅ | Message queue backend provider | -| **topics** | `{ name: string; partitions: number; replicationFactor: number; retentionMs?: number; … }[]` | ✅ | List of topic configurations | -| **consumers** | `{ groupId: string; autoOffsetReset: Enum<'earliest' \| 'latest'>; enableAutoCommit: boolean; maxPollRecords: number }[]` | optional | Consumer group configurations | -| **deadLetterQueue** | `{ enabled: boolean; maxRetries: number; queueName: string }` | optional | Dead letter queue for failed messages | -| **ssl** | `boolean` | ✅ | Enable SSL/TLS for broker connections | -| **sasl** | `{ mechanism: Enum<'plain' \| 'scram-sha-256' \| 'scram-sha-512'>; username: string; password: string }` | optional | SASL authentication configuration | - - ---- - -## MessageQueueProvider - -Supported message queue backend provider - -### Allowed Values - -* `kafka` -* `rabbitmq` -* `aws-sqs` -* `redis-pubsub` -* `google-pubsub` -* `azure-service-bus` - - ---- - -## TopicConfig - -Configuration for a message queue topic - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **name** | `string` | ✅ | Topic name identifier | -| **partitions** | `number` | ✅ | Number of partitions for parallel consumption | -| **replicationFactor** | `number` | ✅ | Number of replicas for fault tolerance | -| **retentionMs** | `number` | optional | Message retention period in milliseconds | -| **compressionType** | `Enum<'none' \| 'gzip' \| 'snappy' \| 'lz4'>` | ✅ | Message compression algorithm | - - ---- - diff --git a/content/docs/references/system/meta.json b/content/docs/references/system/meta.json index fbff8d1e47..d2bd8a631d 100644 --- a/content/docs/references/system/meta.json +++ b/content/docs/references/system/meta.json @@ -19,7 +19,6 @@ "core-services", "http-server", "job", - "message-queue", "metadata-persistence", "notification", "object-storage", diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index a5e4f7c06f..c0b2bc580b 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -406,6 +406,9 @@ ONE AUTHOR-REACHABLE SURFACE reaches this indirectly and is why it is not purely - **`export-field-meta-constraints-retired`** — `@objectstack/rest: ExportFieldMeta.required / .system / .readonly / .hasDefault / .min / .max / .minLength / .maxLength (the map built by `buildFieldMetaMap`, reached as `PreparedImport.metaMap` from `prepareImportRequest`)` → the object schema you already hold — read `fields[name].required` / `.system` / `.readonly` / `.defaultValue` / `.min` / `.max` / `.minLength` / `.maxLength` off the same `ObjectSchema` you passed to `buildFieldMetaMap`, which is where the ENGINE reads them and therefore the only copy that cannot drift - Why not automatic: ADR-0049 enforce-or-remove. These eight were never a source of truth: `buildFieldMetaMap(schema)` DERIVED each one from the very `schema` its caller passed in, so the map carried a second copy of facts the caller already held. They existed for exactly one consumer — the import dry run's hand-copied pre-check mirror (`firstMissingRequiredField` / `firstConstraintViolation`, framework#3956) — and #4633 ruling D retired that mirror (PR #6532): the dry run now asks `DataProtocol.validateData` for the engine's verdict, which reads the object's own schema. That left all eight computed on every import and read by NOTHING, which is the declared-and-unread shape ADR-0049 exists for; a constraint vocabulary standing next to the presentation one with no enforcer behind it is precisely the thing an AI-authored consumer mistakes for a contract. Verified zero-reader before removal, per key and by type, across this repo (`packages/rest` itself, and all five in-repo dependents of `@objectstack/rest`: runtime, cli, verify, plugin-auth, plugin-dev) and the `objectui` sibling; plugin-auth's identity import forwards `prepared.metaMap` into `runImport` but reads only the presentation keys through `coerceRow`. Why this needs a ledger entry despite that sweep: it is the `findStream` (#4484) / `IStorageService.list` (#5540) / `actor-user-roles-to-positions` (#6011) disposition — a published TS surface with NO spec schema, so there is no `retiredKey()` tombstone and no parse rejection that could carry a prescription, and the ledger is the only channel that reaches an upgrader. It is if anything blinder than those three: the keys shipped in a FINAL release (`@objectstack/rest` 14.5.0) and have been published in every release since, and because they were OPTIONAL keys on an interface that itself survives, a JavaScript consumer reading `meta.required` after the upgrade gets `undefined` with no error at all — tsc reports at the read site only for a typed consumer. Why D3 semantic and not a D2 conversion: there is nothing to convert. No authored or stored metadata changes shape — `required` / `min` / `maxLength` and the rest remain fully authorable on a field definition and fully enforced by the engine, which is where they always lived. The only place these eight are ever spelled is inside a consumer's own TypeScript, so no `objectstack migrate meta` transform can reach them. ADR-0049 / ADR-0087, #6536 (the sweep PR #6532 deliberately deferred). - Done when: No code of yours reads any of the eight off a `buildFieldMetaMap` / `prepareImportRequest` result. Grep your sources for `.required` / `.hasDefault` / `.minLength` / `.maxLength` / `.min` / `.max` / `.system` / `.readonly` on an `ExportFieldMeta`-typed value; each hit moves to the object schema you already passed in. ⚠️ Prove it against a RUN, not against tsc: these were optional keys, so an untyped or `any`-typed read compiles clean and silently becomes `undefined` — assert that the constraint your code acts on is still observed on a real import, not merely that the build is green. Note `hasDefault` has no one-to-one replacement key: it was the derived predicate `defaultValue != null`, mirroring the engine's `applyFieldDefaults` gate, so read `fields[name].defaultValue` and apply that same `!= null` test yourself. +- **`external-lookup-message-queue-families-retired`** — `data.externalLookup / data.externalDataSource / data.externalFieldMapping (the whole of data/external-lookup.zod.ts — 3 defs, 8 exported names) and system.messageQueue (the whole of system/message-queue.zod.ts — MessageQueueConfig, MessageQueueProvider, TopicConfig, ConsumerConfig, DeadLetterQueue — 5 defs, 14 exported names)` → (removed — there is no replacement key, because there was never a key: neither family was reachable from any metadata-type binding, stack collection or /meta door, so no document could carry either. For external data: `object.external` (`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference and connection credentials live in the datasource config — never inline in object metadata; `data/external-catalog.zod.ts` is that federated path's catalog surface and is untouched. For message queues: the LIVE surface is `kernel/events/integrations.zod.ts`'s `EventMessageQueueConfig` (`EventBusConfig.messageQueue`), which deliberately carries NO credential field — broker connection and SASL credentials are runtime deployment configuration, not authorable metadata. Either capability returns via the ENFORCE route of ADR-0049 through a new ADR — the executor / broker admin service first, the vocabulary second) + - Why not automatic: Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): security-shaped declared surface with inline-credential sinks and ZERO consumers. `ExternalDataSourceSchema.authentication.config` is a record of unknown whose own docblock example wrote `"clientSecret": "..."` inline, and `MessageQueueConfigSchema.sasl.password` was a required inline broker credential — the #7990 class (cleartext-at-rest credential sinks), except that unlike #7990's two measured surfaces nothing ever persisted these: no metadata-type binding (kernel/metadata-type-schemas.ts imports neither module), no stack collection, no object/field embedding (`object.external` binds `ObjectExternalBindingSchema` — remoteName/remoteSchema/writable/columnMap, no authentication), and zero imports outside packages/spec repo-wide, with the corpus-reach control (`DatasourceSchema` under identical exclusions) returning hits in the same run. The consumed MQ near-namesake `kernel/EventMessageQueueConfig` deliberately has no credential key, so the consumed shape had no credential and the credential-bearing shape had no consumer. A dead schema minus one field is still a dead schema, so the whole declarations go, not just the credential faces (#3950: an exported schema with no consumer reads as a capability to whoever finds it — here it read as an invitation to author secrets in cleartext). With no carrier key there is nothing to tombstone and no source or `sys_metadata` row for a D2 conversion to rewrite: route 3, the #4834 / #4988 / #5055 / #6486 shape — RETIRED_DEFS_BY_MAJOR plus this entry ARE the declaration. ⚠️ The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that retirement's three spellings) is SUBSUMED by the def retirement, the WidgetManifest.performance way: it goes with the shape that carried it. The base `shared/FieldMapping` tombstone and the `integration/ConnectorFieldMapping` spelling are untouched and still reject `transform` with the #5552 prescription. ⚠️ The #7990 Option-B reopen trigger ("a third measured artefact-type surface") is NOT met by this census — that ruling's parked class-level write-boundary guard stays parked; this is the ADR-0049 leg of the fork the triage pre-agreed. + - Done when: No code imports `ExternalLookup(Schema|Parsed)`, `ExternalDataSource(Schema)`, `ExternalFieldMapping(Schema|Parsed)`, `MessageQueueConfig(Schema|Parsed)`, `MessageQueueProvider(Schema)`, `TopicConfig(Schema|Parsed)`, `ConsumerConfig(Schema|Parsed)` or `DeadLetterQueue(Schema|Parsed)` from `@objectstack/spec`, `@objectstack/spec/data` or `@objectstack/spec/system` — every one is TS2305 after upgrade, on every public entry (pinned by resolved symbol identity in `data/external-lookup-retirement.test.ts` and `system/message-queue-retirement.test.ts`). No metadata document needs editing, because none could ever carry one of these shapes. `kernel/EventMessageQueueConfig` (with its inline provider enum and no credential key), `data/external-catalog.zod.ts`, `object.external` and `kernel/DeadLetterQueueEntry` survive unchanged. - **`filter-regex-options-retired`** — `data.filter $regex / $options — in a STORED filter (dashboard widget filter and globalFilters, report runtimeFilter, page and component filter, solution-blueprint filter), and equally in the where clause of a query request` → $icontains for the case-insensitive substring match this was almost always used for, or $contains for a case-sensitive one — a pattern that genuinely needs a regular expression has no filter-level replacement - Why not automatic: Like `driver-aggregate-undeclared-key-aliases-removed` and `driver-sql-distinct-bare-filter-typed`, this entry records a LENIENCY being withdrawn rather than a declared surface: `$regex` was never in `FILTER_OPERATORS` and never a key on `StringOperatorSchema`. That is measured, not assumed — `git log -S'$regex'` over `packages/spec/src` returns only doc comments describing how `$contains` LOWERS to MongoDB (`Contains substring - SQL: LIKE %?% | MongoDB: $regex`), plus #5701 itself, which added the name solely as `RETIRED_FILTER_OPERATORS` prescription data. ⚠️ But it differs from those two in the one way that decides the disposition, so a reader should not have to infer it: those were driver CALL ARGUMENTS, code and never stack metadata, whereas a filter IS stored metadata. `FilterConditionSchema` is an OPEN RECORD (`z.record(z.string(), z.unknown())`) because a filter key is a field name, so a stored `{ name: { $regex: 'acme.*' } }` parses GREEN and always will — a `retiredKey()` tombstone cannot exist on an open map, which is exactly why the ledger has to carry this. What such a stack used to get was four different answers from four backends: `driver-sql` and Turso's remote transport compiled it to a LIKE-escaped SUBSTRING (so `a.b` matched only the literal `a.b` and the regex was silently never a regex), `driver-memory` and objectql's `having` ran it as a real `RegExp` (so the same filter also matched `axb`, and an INVALID pattern was caught and answered `false` — zero rows, in silence), and `driver-mongodb` refused it with a bare `Error` carrying no `code` and no `status`. It is now refused everywhere with INVALID_FILTER / 400 naming the replacement. There is deliberately NO D2 conversion and this sits in `semantic` rather than among the mechanical transforms: rewriting `$regex` to `$icontains` is NOT lossless in either direction — a regex metacharacter becomes a literal — so an auto-applied rewrite would silently change which rows a dashboard, report or permission filter selects, a wrong number rather than a missing one. Choosing the substring the pattern MEANT is a judgment about the query, not a transform. ⚠️ This entry covers BOTH HALVES of the #4706 ruling (B), not just the driver one: the contract half (#5701 — the `$icontains` declaration, the `$contains` family pinned case-sensitive, and the `RETIRED_FILTER_OPERATORS` prescriptions) landed before the ADR-0087 disposition gate (#6148) existed and so was never asked for a ledger entry; the driver half (#5702) is where the refusal became executable. One surface, one entry, registered from the half that made it observable. ADR-0049 / ADR-0087, #4706 / #5701 / #5702. - Done when: No stored filter and no request `where` spells `$regex` or `$options` — grep the stack for both. Each one is rewritten by asking what the pattern MEANT, not by transliterating it: a bare substring pattern becomes `$icontains` (or `$contains` when the match must stay case-sensitive), and its metacharacters are dropped rather than escaped, because they were never honoured as a regex on the SQL family in the first place. ⚠️ Expect the answer to CHANGE on any stack that ran on `driver-memory`, `driver-mongodb` or objectql `having`, where the pattern really was evaluated as a regular expression; on the SQL family the rewritten filter returns what it always returned. A pattern that genuinely needs alternation, anchoring or character classes has no filter-level replacement — move that predicate into a formula field or a server-side view, or open an issue for it. Verify by loading the stack: a surviving `$regex` or `$options` is answered INVALID_FILTER / 400 with a message naming the replacement, on every backend. diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 38cc957e37..e1043dcff8 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -218,17 +218,9 @@ "ExternalColumn (type)", "ExternalColumnParsed (type)", "ExternalColumnSchema (const)", - "ExternalDataSource (type)", - "ExternalDataSourceSchema (const)", "ExternalDatasourceSettings (type)", "ExternalDatasourceSettingsParsed (type)", "ExternalDatasourceSettingsSchema (const)", - "ExternalFieldMapping (type)", - "ExternalFieldMappingParsed (type)", - "ExternalFieldMappingSchema (const)", - "ExternalLookup (type)", - "ExternalLookupParsed (type)", - "ExternalLookupSchema (const)", "ExternalTable (type)", "ExternalTableParsed (type)", "ExternalTableSchema (const)", diff --git a/packages/spec/api-surface/system.json b/packages/spec/api-surface/system.json index aef086ce3d..79324d2b8b 100644 --- a/packages/spec/api-surface/system.json +++ b/packages/spec/api-surface/system.json @@ -136,9 +136,6 @@ "ConsoleDestinationConfig (type)", "ConsoleDestinationConfigParsed (type)", "ConsoleDestinationConfigSchema (const)", - "ConsumerConfig (type)", - "ConsumerConfigParsed (type)", - "ConsumerConfigSchema (const)", "CoreServiceName (type)", "CounterOperation (type)", "CounterOperationSchema (const)", @@ -170,9 +167,6 @@ "DatabaseLevelIsolationStrategySchema (const)", "DatabaseProvider (type)", "DatabaseProviderSchema (const)", - "DeadLetterQueue (type)", - "DeadLetterQueueParsed (type)", - "DeadLetterQueueSchema (const)", "DeleteObjectOperation (type)", "DeployBundle (type)", "DeployBundleParsed (type)", @@ -340,11 +334,6 @@ "MaskingVisibilityRule (type)", "MaskingVisibilityRuleParsed (type)", "MaskingVisibilityRuleSchema (const)", - "MessageQueueConfig (type)", - "MessageQueueConfigParsed (type)", - "MessageQueueConfigSchema (const)", - "MessageQueueProvider (type)", - "MessageQueueProviderSchema (const)", "MetadataCollectionInfo (type)", "MetadataCollectionInfoSchema (const)", "MetadataDiffResult (type)", @@ -699,9 +688,6 @@ "TimeSeriesDataPoint (type)", "TimeSeriesDataPointSchema (const)", "TimeSeriesSchema (const)", - "TopicConfig (type)", - "TopicConfigParsed (type)", - "TopicConfigSchema (const)", "TraceContext (type)", "TraceContextParsed (type)", "TraceContextPropagation (type)", diff --git a/packages/spec/authorable-defaults/data.json b/packages/spec/authorable-defaults/data.json index 57ee560540..6c32c33cd2 100644 --- a/packages/spec/authorable-defaults/data.json +++ b/packages/spec/authorable-defaults/data.json @@ -38,7 +38,6 @@ "data/ExternalDatasourceSettings:allowWrites = false", "data/ExternalDatasourceSettings:queryTimeoutMs = 30000", "data/ExternalDatasourceSettings:validation = {\"checkOnBoot\":true,\"onMismatch\":\"fail\"}", - "data/ExternalFieldMapping:readonly = true", "data/Field:autonumberFormat = \"{0000}\"", "data/Field:deleteBehavior = \"set_null\"", "data/Field:externalId = false", diff --git a/packages/spec/authorable-defaults/system.json b/packages/spec/authorable-defaults/system.json index 06e9380b13..7d90866fec 100644 --- a/packages/spec/authorable-defaults/system.json +++ b/packages/spec/authorable-defaults/system.json @@ -57,9 +57,6 @@ "system/ConsoleDestinationConfig:colors = true", "system/ConsoleDestinationConfig:prettyPrint = false", "system/ConsoleDestinationConfig:stream = \"stdout\"", - "system/ConsumerConfig:autoOffsetReset = \"latest\"", - "system/ConsumerConfig:enableAutoCommit = true", - "system/ConsumerConfig:maxPollRecords = 500", "system/CronSchedule:timezone = \"UTC\"", "system/CursorStyle:opacity = 1", "system/CursorStyle:pulseOnUpdate = true", @@ -67,8 +64,6 @@ "system/DataClassificationPolicy:requireAudit = false", "system/DataClassificationPolicy:requireEncryption = false", "system/DataClassificationPolicy:requireMasking = false", - "system/DeadLetterQueue:enabled = false", - "system/DeadLetterQueue:maxRetries = 3", "system/DeployBundle:flows = []", "system/DeployBundle:objects = []", "system/DeployBundle:permissions = []", @@ -147,7 +142,6 @@ "system/MaskingVisibilityRule:auditUnmask = true", "system/MaskingVisibilityRule:defaultMasked = true", "system/MaskingVisibilityRule:requireApproval = false", - "system/MessageQueueConfig:ssl = false", "system/MetadataHistoryQueryOptions:includeMetadata = true", "system/MetadataHistoryRetentionPolicy:autoCleanup = false", "system/MetadataHistoryRetentionPolicy:cleanupIntervalHours = 24", @@ -253,9 +247,6 @@ "system/TenantUsage:currentStorageBytes = 0", "system/TenantUsage:currentUsers = 0", "system/TenantUsage:deploymentsToday = 0", - "system/TopicConfig:compressionType = \"none\"", - "system/TopicConfig:partitions = 1", - "system/TopicConfig:replicationFactor = 1", "system/TraceContext:remote = false", "system/TraceContext:sampled = true", "system/TraceContext:traceFlags = 1", diff --git a/packages/spec/authorable-surface/data.json b/packages/spec/authorable-surface/data.json index b5c17f27d5..3798cc954f 100644 --- a/packages/spec/authorable-surface/data.json +++ b/packages/spec/authorable-surface/data.json @@ -320,32 +320,11 @@ "data/ExternalColumn:primaryKey", "data/ExternalColumn:sqlType", "data/ExternalColumn:suggestedFieldType", - "data/ExternalDataSource:authentication", - "data/ExternalDataSource:endpoint", - "data/ExternalDataSource:id", - "data/ExternalDataSource:name", - "data/ExternalDataSource:type", "data/ExternalDatasourceSettings:allowWrites", "data/ExternalDatasourceSettings:allowedSchemas", "data/ExternalDatasourceSettings:credentialsRef", "data/ExternalDatasourceSettings:queryTimeoutMs", "data/ExternalDatasourceSettings:validation", - "data/ExternalFieldMapping:defaultValue", - "data/ExternalFieldMapping:readonly", - "data/ExternalFieldMapping:source", - "data/ExternalFieldMapping:target", - "data/ExternalFieldMapping:transform [RETIRED]", - "data/ExternalFieldMapping:type", - "data/ExternalLookup:caching", - "data/ExternalLookup:dataSource", - "data/ExternalLookup:fallback", - "data/ExternalLookup:fieldMappings", - "data/ExternalLookup:fieldName", - "data/ExternalLookup:pagination", - "data/ExternalLookup:query", - "data/ExternalLookup:rateLimit", - "data/ExternalLookup:retry", - "data/ExternalLookup:transform", "data/ExternalTable:columns", "data/ExternalTable:indexes", "data/ExternalTable:remoteName", diff --git a/packages/spec/authorable-surface/system.json b/packages/spec/authorable-surface/system.json index 08550612d5..34ae44afd5 100644 --- a/packages/spec/authorable-surface/system.json +++ b/packages/spec/authorable-surface/system.json @@ -259,10 +259,6 @@ "system/ConsoleDestinationConfig:colors", "system/ConsoleDestinationConfig:prettyPrint", "system/ConsoleDestinationConfig:stream", - "system/ConsumerConfig:autoOffsetReset", - "system/ConsumerConfig:enableAutoCommit", - "system/ConsumerConfig:groupId", - "system/ConsumerConfig:maxPollRecords", "system/CounterOperation:delta", "system/CounterOperation:replicaId", "system/CounterOperation:timestamp", @@ -306,9 +302,6 @@ "system/DatabaseLevelIsolationStrategy:database", "system/DatabaseLevelIsolationStrategy:encryption", "system/DatabaseLevelIsolationStrategy:strategy", - "system/DeadLetterQueue:enabled", - "system/DeadLetterQueue:maxRetries", - "system/DeadLetterQueue:queueName", "system/DeleteObjectOperation:objectName", "system/DeleteObjectOperation:type", "system/DeployBundle:flows", @@ -628,12 +621,6 @@ "system/MaskingVisibilityRule:defaultMasked", "system/MaskingVisibilityRule:requireApproval", "system/MaskingVisibilityRule:unmaskRoles", - "system/MessageQueueConfig:consumers", - "system/MessageQueueConfig:deadLetterQueue", - "system/MessageQueueConfig:provider", - "system/MessageQueueConfig:sasl", - "system/MessageQueueConfig:ssl", - "system/MessageQueueConfig:topics", "system/MetadataCollectionInfo:count", "system/MetadataCollectionInfo:namespaces", "system/MetadataCollectionInfo:type", @@ -1265,11 +1252,6 @@ "system/TimeSeriesDataPoint:labels", "system/TimeSeriesDataPoint:timestamp", "system/TimeSeriesDataPoint:value", - "system/TopicConfig:compressionType", - "system/TopicConfig:name", - "system/TopicConfig:partitions", - "system/TopicConfig:replicationFactor", - "system/TopicConfig:retentionMs", "system/TraceContext:parentSpanId", "system/TraceContext:remote", "system/TraceContext:sampled", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index 074d214cd3..46753b9b51 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -218,17 +218,9 @@ "ExternalColumn": "src/data/external-catalog.zod.ts#ExternalColumn (type)", "ExternalColumnParsed": "src/data/external-catalog.zod.ts#ExternalColumnParsed (type)", "ExternalColumnSchema": "src/data/external-catalog.zod.ts#ExternalColumnSchema (const)", - "ExternalDataSource": "src/data/external-lookup.zod.ts#ExternalDataSource (type)", - "ExternalDataSourceSchema": "src/data/external-lookup.zod.ts#ExternalDataSourceSchema (const)", "ExternalDatasourceSettings": "src/data/datasource.zod.ts#ExternalDatasourceSettings (type)", "ExternalDatasourceSettingsParsed": "src/data/datasource.zod.ts#ExternalDatasourceSettingsParsed (type)", "ExternalDatasourceSettingsSchema": "src/data/datasource.zod.ts#ExternalDatasourceSettingsSchema (const)", - "ExternalFieldMapping": "src/data/external-lookup.zod.ts#ExternalFieldMapping (type)", - "ExternalFieldMappingParsed": "src/data/external-lookup.zod.ts#ExternalFieldMappingParsed (type)", - "ExternalFieldMappingSchema": "src/data/external-lookup.zod.ts#ExternalFieldMappingSchema (const)", - "ExternalLookup": "src/data/external-lookup.zod.ts#ExternalLookup (type)", - "ExternalLookupParsed": "src/data/external-lookup.zod.ts#ExternalLookupParsed (type)", - "ExternalLookupSchema": "src/data/external-lookup.zod.ts#ExternalLookupSchema (const)", "ExternalTable": "src/data/external-catalog.zod.ts#ExternalTable (type)", "ExternalTableParsed": "src/data/external-catalog.zod.ts#ExternalTableParsed (type)", "ExternalTableSchema": "src/data/external-catalog.zod.ts#ExternalTableSchema (const)", diff --git a/packages/spec/export-origins/system.json b/packages/spec/export-origins/system.json index 8be122e10c..80c314be33 100644 --- a/packages/spec/export-origins/system.json +++ b/packages/spec/export-origins/system.json @@ -136,9 +136,6 @@ "ConsoleDestinationConfig": "src/system/logging.zod.ts#ConsoleDestinationConfig (type)", "ConsoleDestinationConfigParsed": "src/system/logging.zod.ts#ConsoleDestinationConfigParsed (type)", "ConsoleDestinationConfigSchema": "src/system/logging.zod.ts#ConsoleDestinationConfigSchema (const)", - "ConsumerConfig": "src/system/message-queue.zod.ts#ConsumerConfig (type)", - "ConsumerConfigParsed": "src/system/message-queue.zod.ts#ConsumerConfigParsed (type)", - "ConsumerConfigSchema": "src/system/message-queue.zod.ts#ConsumerConfigSchema (const)", "CoreServiceName": "src/system/core-services.zod.ts#CoreServiceName (type)", "CounterOperation": "src/system/collaboration.zod.ts#CounterOperation (type)", "CounterOperationSchema": "src/system/collaboration.zod.ts#CounterOperationSchema (const)", @@ -170,9 +167,6 @@ "DatabaseLevelIsolationStrategySchema": "src/system/tenant.zod.ts#DatabaseLevelIsolationStrategySchema (const)", "DatabaseProvider": "src/system/tenant.zod.ts#DatabaseProvider (type)", "DatabaseProviderSchema": "src/system/tenant.zod.ts#DatabaseProviderSchema (const)", - "DeadLetterQueue": "src/system/message-queue.zod.ts#DeadLetterQueue (type)", - "DeadLetterQueueParsed": "src/system/message-queue.zod.ts#DeadLetterQueueParsed (type)", - "DeadLetterQueueSchema": "src/system/message-queue.zod.ts#DeadLetterQueueSchema (const)", "DeleteObjectOperation": "src/system/migration.zod.ts#DeleteObjectOperation (type)", "DeployBundle": "src/system/deploy-bundle.zod.ts#DeployBundle (type)", "DeployBundleParsed": "src/system/deploy-bundle.zod.ts#DeployBundleParsed (type)", @@ -340,11 +334,6 @@ "MaskingVisibilityRule": "src/system/security-context.zod.ts#MaskingVisibilityRule (type)", "MaskingVisibilityRuleParsed": "src/system/security-context.zod.ts#MaskingVisibilityRuleParsed (type)", "MaskingVisibilityRuleSchema": "src/system/security-context.zod.ts#MaskingVisibilityRuleSchema (const)", - "MessageQueueConfig": "src/system/message-queue.zod.ts#MessageQueueConfig (type)", - "MessageQueueConfigParsed": "src/system/message-queue.zod.ts#MessageQueueConfigParsed (type)", - "MessageQueueConfigSchema": "src/system/message-queue.zod.ts#MessageQueueConfigSchema (const)", - "MessageQueueProvider": "src/system/message-queue.zod.ts#MessageQueueProvider (type)", - "MessageQueueProviderSchema": "src/system/message-queue.zod.ts#MessageQueueProviderSchema (const)", "MetadataCollectionInfo": "src/system/metadata-persistence.zod.ts#MetadataCollectionInfo (type)", "MetadataCollectionInfoSchema": "src/system/metadata-persistence.zod.ts#MetadataCollectionInfoSchema (const)", "MetadataDiffResult": "src/system/metadata-persistence.zod.ts#MetadataDiffResult (type)", @@ -699,9 +688,6 @@ "TimeSeriesDataPoint": "src/system/metrics.zod.ts#TimeSeriesDataPoint (type)", "TimeSeriesDataPointSchema": "src/system/metrics.zod.ts#TimeSeriesDataPointSchema (const)", "TimeSeriesSchema": "src/system/metrics.zod.ts#TimeSeriesSchema (const)", - "TopicConfig": "src/system/message-queue.zod.ts#TopicConfig (type)", - "TopicConfigParsed": "src/system/message-queue.zod.ts#TopicConfigParsed (type)", - "TopicConfigSchema": "src/system/message-queue.zod.ts#TopicConfigSchema (const)", "TraceContext": "src/system/tracing.zod.ts#TraceContext (type)", "TraceContextParsed": "src/system/tracing.zod.ts#TraceContextParsed (type)", "TraceContextPropagation": "src/system/tracing.zod.ts#TraceContextPropagation (type)", diff --git a/packages/spec/json-schema.manifest/data.json b/packages/spec/json-schema.manifest/data.json index f5b63933bf..58e03a5eb3 100644 --- a/packages/spec/json-schema.manifest/data.json +++ b/packages/spec/json-schema.manifest/data.json @@ -71,10 +71,7 @@ "data/ExpressionBody", "data/ExternalCatalog", "data/ExternalColumn", - "data/ExternalDataSource", "data/ExternalDatasourceSettings", - "data/ExternalFieldMapping", - "data/ExternalLookup", "data/ExternalTable", "data/FeedFilterMode", "data/FeedItemType", diff --git a/packages/spec/json-schema.manifest/system.json b/packages/spec/json-schema.manifest/system.json index 640169f655..df910b6964 100644 --- a/packages/spec/json-schema.manifest/system.json +++ b/packages/spec/json-schema.manifest/system.json @@ -52,7 +52,6 @@ "system/ComplianceEncryptionRequirement", "system/ComplianceFramework", "system/ConsoleDestinationConfig", - "system/ConsumerConfig", "system/CoreServiceName", "system/CounterOperation", "system/CoverageBreakdownEntry", @@ -67,7 +66,6 @@ "system/DataMigrationFlag", "system/DatabaseLevelIsolationStrategy", "system/DatabaseProvider", - "system/DeadLetterQueue", "system/DeleteObjectOperation", "system/DeployBundle", "system/DeployDiff", @@ -134,8 +132,6 @@ "system/LoggerConfig", "system/LoggingConfig", "system/MaskingVisibilityRule", - "system/MessageQueueConfig", - "system/MessageQueueProvider", "system/MetadataCollectionInfo", "system/MetadataDiffResult", "system/MetadataFallbackStrategy", @@ -271,7 +267,6 @@ "system/TextCRDTState", "system/TimeSeries", "system/TimeSeriesDataPoint", - "system/TopicConfig", "system/TraceContext", "system/TraceContextPropagation", "system/TraceFlags", diff --git a/packages/spec/scripts/build-docs.ts b/packages/spec/scripts/build-docs.ts index 738baadd1d..eb7553deab 100644 --- a/packages/spec/scripts/build-docs.ts +++ b/packages/spec/scripts/build-docs.ts @@ -494,7 +494,9 @@ const SECTION_GROUPS: Record data: [ { section: 'Objects & Fields', pages: ['object', 'field', 'validation', 'hook', 'hook-body', 'mapping'] }, { section: 'Query & Analytics', pages: ['query', 'filter', 'data-engine', 'analytics', 'date-macros'] }, - { section: 'Datasources & Drivers', pages: ['datasource', 'driver', 'driver-sql', 'driver-nosql', 'external-catalog', 'external-lookup'] }, + // `external-lookup` retired whole at #8075 (ADR-0049) — `external-catalog` + // is the surviving federated-external-data surface. + { section: 'Datasources & Drivers', pages: ['datasource', 'driver', 'driver-sql', 'driver-nosql', 'external-catalog'] }, { section: 'Documents & Seed', pages: ['document', 'seed', 'seed-loader', 'feed'] }, ], integration: [ @@ -516,7 +518,9 @@ const SECTION_GROUPS: Record // `metadata-loader` removed at #4696 — that file lives in `kernel/`, and the // two schemas `system/` re-exports from it are documented on // `system/metadata-persistence`, the file that re-exports them. - { section: 'Services & Infrastructure', pages: ['core-services', 'http-server', 'cache', 'message-queue', 'object-storage', 'search-engine', 'worker', 'job', 'notification', 'translation', 'metadata-persistence'] }, + // `message-queue` retired whole at #8075 (ADR-0049) — the live MQ surface + // is kernel's `EventMessageQueueConfig`, documented with the event bus. + { section: 'Services & Infrastructure', pages: ['core-services', 'http-server', 'cache', 'object-storage', 'search-engine', 'worker', 'job', 'notification', 'translation', 'metadata-persistence'] }, { section: 'Observability', pages: ['logging', 'metrics', 'tracing', 'audit'] }, { section: 'Security & Compliance', pages: ['encryption', 'security-context', 'incident-response', 'supplier-security', 'disaster-recovery', 'change-management', 'training'] }, { section: 'Content & Collaboration', pages: ['doc', 'book', 'collaboration'] }, diff --git a/packages/spec/scripts/lib/renamed-defs.ts b/packages/spec/scripts/lib/renamed-defs.ts index e240bb0846..5153357a96 100644 --- a/packages/spec/scripts/lib/renamed-defs.ts +++ b/packages/spec/scripts/lib/renamed-defs.ts @@ -74,7 +74,8 @@ export const RENAMED_DEFS: Readonly> = { // #4703 / ADR-0112 D9a — `FieldMapping` was published by THREE defs at once. // The two domain-specific sides take a domain prefix; `shared/FieldMapping` - // is the BASE that this rename's target (and `data/ExternalFieldMapping`) + // is the BASE that this rename's target (and, until #8075's whole-family + // retirement, `data/ExternalFieldMapping`) // extend, so it keeps the bare name and is deliberately absent from this // table. Note what an `extend` means for the invariants below: the target // def's key set is a superset of the base's, so every carried key is found, diff --git a/packages/spec/scripts/renamed-defs.test.ts b/packages/spec/scripts/renamed-defs.test.ts index 265a100d9a..47d746ed94 100644 --- a/packages/spec/scripts/renamed-defs.test.ts +++ b/packages/spec/scripts/renamed-defs.test.ts @@ -169,10 +169,10 @@ describe('the committed RENAMED_DEFS table', () => { }); it('leaves the shared BASE alone — `shared/FieldMapping` keeps the bare name', () => { - // `integration/ConnectorFieldMapping` and `data/ExternalFieldMapping` both - // `.extend()` it. Renaming the base would move keys under two other defs - // and change nothing about the collision, which was between the two - // domain-specific sides and the base's own name. + // `integration/ConnectorFieldMapping` (and `data/ExternalFieldMapping`, + // until #8075 retired that family whole) `.extend()` it. Renaming the base + // would move keys under other defs and change nothing about the collision, + // which was between the two domain-specific sides and the base's own name. expect(RENAMED_DEFS['shared/FieldMapping']).toBeUndefined(); }); diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index ad0ad39fb8..1aeaadbb0a 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -693,6 +693,13 @@ "toMajor": 17, "rationale": "ADR-0049 enforce-or-remove. These eight were never a source of truth: `buildFieldMetaMap(schema)` DERIVED each one from the very `schema` its caller passed in, so the map carried a second copy of facts the caller already held. They existed for exactly one consumer — the import dry run's hand-copied pre-check mirror (`firstMissingRequiredField` / `firstConstraintViolation`, framework#3956) — and #4633 ruling D retired that mirror (PR #6532): the dry run now asks `DataProtocol.validateData` for the engine's verdict, which reads the object's own schema. That left all eight computed on every import and read by NOTHING, which is the declared-and-unread shape ADR-0049 exists for; a constraint vocabulary standing next to the presentation one with no enforcer behind it is precisely the thing an AI-authored consumer mistakes for a contract. Verified zero-reader before removal, per key and by type, across this repo (`packages/rest` itself, and all five in-repo dependents of `@objectstack/rest`: runtime, cli, verify, plugin-auth, plugin-dev) and the `objectui` sibling; plugin-auth's identity import forwards `prepared.metaMap` into `runImport` but reads only the presentation keys through `coerceRow`. Why this needs a ledger entry despite that sweep: it is the `findStream` (#4484) / `IStorageService.list` (#5540) / `actor-user-roles-to-positions` (#6011) disposition — a published TS surface with NO spec schema, so there is no `retiredKey()` tombstone and no parse rejection that could carry a prescription, and the ledger is the only channel that reaches an upgrader. It is if anything blinder than those three: the keys shipped in a FINAL release (`@objectstack/rest` 14.5.0) and have been published in every release since, and because they were OPTIONAL keys on an interface that itself survives, a JavaScript consumer reading `meta.required` after the upgrade gets `undefined` with no error at all — tsc reports at the read site only for a typed consumer. Why D3 semantic and not a D2 conversion: there is nothing to convert. No authored or stored metadata changes shape — `required` / `min` / `maxLength` and the rest remain fully authorable on a field definition and fully enforced by the engine, which is where they always lived. The only place these eight are ever spelled is inside a consumer's own TypeScript, so no `objectstack migrate meta` transform can reach them. ADR-0049 / ADR-0087, #6536 (the sweep PR #6532 deliberately deferred)." }, + { + "surface": "data.externalLookup / data.externalDataSource / data.externalFieldMapping (the whole of data/external-lookup.zod.ts — 3 defs, 8 exported names) and system.messageQueue (the whole of system/message-queue.zod.ts — MessageQueueConfig, MessageQueueProvider, TopicConfig, ConsumerConfig, DeadLetterQueue — 5 defs, 14 exported names)", + "replacement": "(removed — there is no replacement key, because there was never a key: neither family was reachable from any metadata-type binding, stack collection or /meta door, so no document could carry either. For external data: `object.external` (`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference and connection credentials live in the datasource config — never inline in object metadata; `data/external-catalog.zod.ts` is that federated path's catalog surface and is untouched. For message queues: the LIVE surface is `kernel/events/integrations.zod.ts`'s `EventMessageQueueConfig` (`EventBusConfig.messageQueue`), which deliberately carries NO credential field — broker connection and SASL credentials are runtime deployment configuration, not authorable metadata. Either capability returns via the ENFORCE route of ADR-0049 through a new ADR — the executor / broker admin service first, the vocabulary second)", + "migrationId": "external-lookup-message-queue-families-retired", + "toMajor": 17, + "rationale": "Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): security-shaped declared surface with inline-credential sinks and ZERO consumers. `ExternalDataSourceSchema.authentication.config` is a record of unknown whose own docblock example wrote `\"clientSecret\": \"...\"` inline, and `MessageQueueConfigSchema.sasl.password` was a required inline broker credential — the #7990 class (cleartext-at-rest credential sinks), except that unlike #7990's two measured surfaces nothing ever persisted these: no metadata-type binding (kernel/metadata-type-schemas.ts imports neither module), no stack collection, no object/field embedding (`object.external` binds `ObjectExternalBindingSchema` — remoteName/remoteSchema/writable/columnMap, no authentication), and zero imports outside packages/spec repo-wide, with the corpus-reach control (`DatasourceSchema` under identical exclusions) returning hits in the same run. The consumed MQ near-namesake `kernel/EventMessageQueueConfig` deliberately has no credential key, so the consumed shape had no credential and the credential-bearing shape had no consumer. A dead schema minus one field is still a dead schema, so the whole declarations go, not just the credential faces (#3950: an exported schema with no consumer reads as a capability to whoever finds it — here it read as an invitation to author secrets in cleartext). With no carrier key there is nothing to tombstone and no source or `sys_metadata` row for a D2 conversion to rewrite: route 3, the #4834 / #4988 / #5055 / #6486 shape — RETIRED_DEFS_BY_MAJOR plus this entry ARE the declaration. ⚠️ The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that retirement's three spellings) is SUBSUMED by the def retirement, the WidgetManifest.performance way: it goes with the shape that carried it. The base `shared/FieldMapping` tombstone and the `integration/ConnectorFieldMapping` spelling are untouched and still reject `transform` with the #5552 prescription. ⚠️ The #7990 Option-B reopen trigger (\"a third measured artefact-type surface\") is NOT met by this census — that ruling's parked class-level write-boundary guard stays parked; this is the ADR-0049 leg of the fork the triage pre-agreed." + }, { "surface": "data.filter $regex / $options — in a STORED filter (dashboard widget filter and globalFilters, report runtimeFilter, page and component filter, solution-blueprint filter), and equally in the where clause of a query request", "replacement": "$icontains for the case-insensitive substring match this was almost always used for, or $contains for a case-sensitive one — a pattern that genuinely needs a regular expression has no filter-level replacement", @@ -1675,6 +1682,13 @@ "toMajor": 17, "rationale": "ADR-0049 enforce-or-remove. These eight were never a source of truth: `buildFieldMetaMap(schema)` DERIVED each one from the very `schema` its caller passed in, so the map carried a second copy of facts the caller already held. They existed for exactly one consumer — the import dry run's hand-copied pre-check mirror (`firstMissingRequiredField` / `firstConstraintViolation`, framework#3956) — and #4633 ruling D retired that mirror (PR #6532): the dry run now asks `DataProtocol.validateData` for the engine's verdict, which reads the object's own schema. That left all eight computed on every import and read by NOTHING, which is the declared-and-unread shape ADR-0049 exists for; a constraint vocabulary standing next to the presentation one with no enforcer behind it is precisely the thing an AI-authored consumer mistakes for a contract. Verified zero-reader before removal, per key and by type, across this repo (`packages/rest` itself, and all five in-repo dependents of `@objectstack/rest`: runtime, cli, verify, plugin-auth, plugin-dev) and the `objectui` sibling; plugin-auth's identity import forwards `prepared.metaMap` into `runImport` but reads only the presentation keys through `coerceRow`. Why this needs a ledger entry despite that sweep: it is the `findStream` (#4484) / `IStorageService.list` (#5540) / `actor-user-roles-to-positions` (#6011) disposition — a published TS surface with NO spec schema, so there is no `retiredKey()` tombstone and no parse rejection that could carry a prescription, and the ledger is the only channel that reaches an upgrader. It is if anything blinder than those three: the keys shipped in a FINAL release (`@objectstack/rest` 14.5.0) and have been published in every release since, and because they were OPTIONAL keys on an interface that itself survives, a JavaScript consumer reading `meta.required` after the upgrade gets `undefined` with no error at all — tsc reports at the read site only for a typed consumer. Why D3 semantic and not a D2 conversion: there is nothing to convert. No authored or stored metadata changes shape — `required` / `min` / `maxLength` and the rest remain fully authorable on a field definition and fully enforced by the engine, which is where they always lived. The only place these eight are ever spelled is inside a consumer's own TypeScript, so no `objectstack migrate meta` transform can reach them. ADR-0049 / ADR-0087, #6536 (the sweep PR #6532 deliberately deferred)." }, + { + "surface": "data.externalLookup / data.externalDataSource / data.externalFieldMapping (the whole of data/external-lookup.zod.ts — 3 defs, 8 exported names) and system.messageQueue (the whole of system/message-queue.zod.ts — MessageQueueConfig, MessageQueueProvider, TopicConfig, ConsumerConfig, DeadLetterQueue — 5 defs, 14 exported names)", + "replacement": "(removed — there is no replacement key, because there was never a key: neither family was reachable from any metadata-type binding, stack collection or /meta door, so no document could carry either. For external data: `object.external` (`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference and connection credentials live in the datasource config — never inline in object metadata; `data/external-catalog.zod.ts` is that federated path's catalog surface and is untouched. For message queues: the LIVE surface is `kernel/events/integrations.zod.ts`'s `EventMessageQueueConfig` (`EventBusConfig.messageQueue`), which deliberately carries NO credential field — broker connection and SASL credentials are runtime deployment configuration, not authorable metadata. Either capability returns via the ENFORCE route of ADR-0049 through a new ADR — the executor / broker admin service first, the vocabulary second)", + "migrationId": "external-lookup-message-queue-families-retired", + "toMajor": 17, + "rationale": "Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): security-shaped declared surface with inline-credential sinks and ZERO consumers. `ExternalDataSourceSchema.authentication.config` is a record of unknown whose own docblock example wrote `\"clientSecret\": \"...\"` inline, and `MessageQueueConfigSchema.sasl.password` was a required inline broker credential — the #7990 class (cleartext-at-rest credential sinks), except that unlike #7990's two measured surfaces nothing ever persisted these: no metadata-type binding (kernel/metadata-type-schemas.ts imports neither module), no stack collection, no object/field embedding (`object.external` binds `ObjectExternalBindingSchema` — remoteName/remoteSchema/writable/columnMap, no authentication), and zero imports outside packages/spec repo-wide, with the corpus-reach control (`DatasourceSchema` under identical exclusions) returning hits in the same run. The consumed MQ near-namesake `kernel/EventMessageQueueConfig` deliberately has no credential key, so the consumed shape had no credential and the credential-bearing shape had no consumer. A dead schema minus one field is still a dead schema, so the whole declarations go, not just the credential faces (#3950: an exported schema with no consumer reads as a capability to whoever finds it — here it read as an invitation to author secrets in cleartext). With no carrier key there is nothing to tombstone and no source or `sys_metadata` row for a D2 conversion to rewrite: route 3, the #4834 / #4988 / #5055 / #6486 shape — RETIRED_DEFS_BY_MAJOR plus this entry ARE the declaration. ⚠️ The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that retirement's three spellings) is SUBSUMED by the def retirement, the WidgetManifest.performance way: it goes with the shape that carried it. The base `shared/FieldMapping` tombstone and the `integration/ConnectorFieldMapping` spelling are untouched and still reject `transform` with the #5552 prescription. ⚠️ The #7990 Option-B reopen trigger (\"a third measured artefact-type surface\") is NOT met by this census — that ruling's parked class-level write-boundary guard stays parked; this is the ADR-0049 leg of the fork the triage pre-agreed." + }, { "surface": "data.filter $regex / $options — in a STORED filter (dashboard widget filter and globalFilters, report runtimeFilter, page and component filter, solution-blueprint filter), and equally in the where clause of a query request", "replacement": "$icontains for the case-insensitive substring match this was almost always used for, or $contains for a case-sensitive one — a pattern that genuinely needs a regular expression has no filter-level replacement", diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index 68be2f669b..014a92ff45 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -4468,7 +4468,14 @@ const connectorRateLimitConfigRemoved: MetadataConversion = { * source that can carry the key. `ExternalLookupSchema` is not referenced by * any stack collection or metadata type, so there is no external-lookup * document for the walker to visit; its authorable key is retired by the same - * tombstone and needs no transform. + * tombstone and needs no transform. (That zero-consumer finding then became + * the whole family's verdict: #8075 retired `data/external-lookup.zod.ts` + * outright — route 3, D3 `external-lookup-message-queue-families-retired` — + * and the `data/ExternalFieldMapping:transform` retired-keys entry was + * subsumed by the def retirement, the WidgetManifest.performance way. The + * `externalLookup.…` clause in this entry's `surface` stays: it correctly + * documents what the v16→17 change list removed, and it composes with the def + * retirement into "delete the whole value".) * * `retiredFromLoadPath`: the key claims a transformation that never ran, the * `connector-rate-limit-config-removed` shape exactly. Absorbing it silently at diff --git a/packages/spec/src/data/external-lookup-retirement.test.ts b/packages/spec/src/data/external-lookup-retirement.test.ts new file mode 100644 index 0000000000..4b9e1e0e41 --- /dev/null +++ b/packages/spec/src/data/external-lookup-retirement.test.ts @@ -0,0 +1,161 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; + +import { + EXPORT_ENTRY_POINTS, + exportNamesOf, + holdersOf, +} from '../../scripts/lib/export-origins-testkit'; + +// ─── [#8075] the external-lookup family is RETIRED ────────────────────────── +// +// ADR-0049 enforce-or-remove, fork (b) of the #8075 census (accepted +// 2026-08-12): `data/external-lookup.zod.ts` is deleted whole — 3 emitted defs +// (`data/ExternalDataSource`, `data/ExternalFieldMapping`, +// `data/ExternalLookup`), 8 exported names, reference docs with them. +// +// The measurement that decided it (issue #8075, report comment; spot-verified +// on the merged tree before this removal, control passing in the SAME run): +// +// 1. STATIC — zero imports of any export outside `packages/spec` repo-wide, +// while the corpus-reach control (`DatasourceSchema` under identical +// exclusions) returns hits. In-package, the only non-test consumer of +// `ExternalDataSourceSchema` / `ExternalFieldMappingSchema` was +// `ExternalLookupSchema` in the same module, itself consumed by nothing. +// 2. DOORS — no metadata-type binding (kernel/metadata-type-schemas.ts +// imports neither module), no stack collection, no object/field +// embedding: `object.external` binds `ObjectExternalBindingSchema` +// (remoteName / remoteSchema / writable / columnMap — no authentication; +// the ADR-0015/0062 federated path routes credentials through datasource +// config). The #5552 conversion's docblock had already recorded that no +// external-lookup document exists for the conversion walker to visit. +// 3. The security face: `ExternalDataSourceSchema.authentication.config` +// was a `z.record(z.string(), z.unknown())` whose own docblock example +// wrote `"clientSecret": "..."` inline — an invitation to author OAuth +// secrets in cleartext metadata, with no consumer to ever read them. +// +// ## Why route 3, and why there is nothing to tombstone +// +// With no carrier key there is no shape on which a `retiredKey()` tombstone +// could sit, and no author document for an ADR-0087 D2 conversion to rewrite — +// a prescription nobody can receive is noise. The declared record is the D3 +// `SemanticMigration` `external-lookup-message-queue-families-retired` plus +// the `RETIRED_DEFS_BY_MAJOR[17]` entries the manifest-deletion gate reads. +// +// The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that +// retirement's three spellings) is SUBSUMED rather than deleted-in-isolation: +// it goes with the shape that carried it, which is strictly stronger, because +// there is no longer a mapping shape to author the key INTO. The base +// `shared/FieldMapping` tombstone and the `integration/ConnectorFieldMapping` +// spelling are untouched — `shared/mapping.test.ts` still pins those. +// +// Form follows #4988 / #5055: resolved symbol identity over every public entry +// via the build-time `export-origins/` artifact, plus the file-deletion probe +// in the #4988 direction (whole-file retirement, no surviving occupant). +describe('[#8075] data/ external-lookup family retirement', () => { + /** The 8 names the three retired defs exported (3 schema consts + 5 types). */ + const RETIRED_NAMES = [ + 'ExternalDataSourceSchema', 'ExternalDataSource', + 'ExternalFieldMappingSchema', 'ExternalFieldMapping', 'ExternalFieldMappingParsed', + 'ExternalLookupSchema', 'ExternalLookup', 'ExternalLookupParsed', + ] as const; + + /** + * Names that must SURVIVE on `./data`. The first three are the live + * federated-external-data path the retirement's prescription points at; the + * catalog trio lives one file over from the deleted module and is exactly + * what a too-wide "finish everything external" sweep would take. + */ + const MUST_SURVIVE_DATA = [ + 'ObjectExternalBindingSchema', + 'DatasourceSchema', + 'ImportFieldMappingSchema', + 'ExternalCatalogSchema', + 'ExternalTableSchema', + 'ExternalColumnSchema', + ] as const; + + it('every retired name has ZERO holders on any public entry; the survivors still stand', () => { + // Anti-vacuity: the baseline must cover the real surface. + for (const needed of ['.', './data', './shared', './system']) { + expect(EXPORT_ENTRY_POINTS, `exports map must include ${needed}`).toContain(needed); + } + expect(exportNamesOf('./data').length, './data must export a non-trivial surface').toBeGreaterThan(100); + + // ── ABSENCE (every entry, not just ./data) ──────────────────────────── + for (const name of RETIRED_NAMES) { + expect(holdersOf(name), `${name} must have zero holders after #8075`).toEqual([]); + } + + // ── SURVIVAL ────────────────────────────────────────────────────────── + const dataNames = exportNamesOf('./data'); + for (const name of MUST_SURVIVE_DATA) { + expect(dataNames, `${name} must SURVIVE this retirement`).toContain(name); + } + // The base the deleted `ExternalFieldMappingSchema` used to extend keeps + // its bare name on ./shared — the retirement takes the extender, never + // the base. + expect(exportNamesOf('./shared')).toContain('FieldMappingSchema'); + }); + + it('the module is gone from disk, and nothing imports it any more', async () => { + const fs = await import('node:fs'); + const path = await import('node:path'); + const { fileURLToPath } = await import('node:url'); + const srcRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); + + for (const f of ['external-lookup.zod.ts', 'external-lookup.test.ts']) { + expect(fs.existsSync(path.join(srcRoot, 'data', f)), `data/${f} must be deleted`).toBe(false); + } + // Anti-vacuity: the sibling that was measured live-adjacent and KEPT must + // still be on disk, so "false" above cannot mean "wrong directory". + expect(fs.existsSync(path.join(srcRoot, 'data', 'external-catalog.zod.ts'))).toBe(true); + + const importers: string[] = []; + const walk = (dir: string) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) walk(full); + else if (entry.name.endsWith('.ts')) { + const src = fs.readFileSync(full, 'utf-8'); + if (/(?:import|export)[^;]*['"][^'"]*\/external-lookup\.zod(?:\.js)?['"]/.test(src)) { + importers.push(path.relative(srcRoot, full)); + } + } + } + }; + walk(srcRoot); + expect(importers, 'a resurrected import means the retirement is being undone — re-read #8075').toEqual([]); + }); + + it('runtime namespace agrees with the compiler view', async () => { + const data = await import('./index'); + for (const name of RETIRED_NAMES) { + expect(name in data, `data must not export ${name}`).toBe(false); + } + for (const name of MUST_SURVIVE_DATA) { + expect(name in data, `${name} must SURVIVE at runtime`).toBe(true); + } + }); + + it('the live external path still carries NO inline credential face', async () => { + // The retirement's argument in one assertion: the surviving federated + // binding (`object.external`) exposes no `authentication` and no secret + // slot — credentials belong to datasource config, not object metadata. If + // someone re-adds an inline credential face here, this pin asks for the + // #7990 / #8075 analysis to be re-run, not for a quiet green. + const { ObjectExternalBindingSchema } = await import('./object.zod'); + const parsed = ObjectExternalBindingSchema.safeParse({ + datasource: 'warehouse', + remoteName: 'accounts', + authentication: { type: 'oauth2', config: { clientSecret: 'x' } }, + }); + // strictObject: an inline `authentication` block is REJECTED, not stored. + expect(parsed.success).toBe(false); + if (!parsed.success) { + expect(parsed.error.issues.some((i) => i.code === 'unrecognized_keys' + && (i as { keys?: string[] }).keys?.includes('authentication'))).toBe(true); + } + }); +}); diff --git a/packages/spec/src/data/external-lookup.test.ts b/packages/spec/src/data/external-lookup.test.ts deleted file mode 100644 index d33ab6ba73..0000000000 --- a/packages/spec/src/data/external-lookup.test.ts +++ /dev/null @@ -1,817 +0,0 @@ -import { describe, it, expect } from 'vitest'; -import { - ExternalDataSourceSchema, - ExternalFieldMappingSchema, - ExternalLookupSchema, - type ExternalLookup, - type ExternalDataSource, - type ExternalFieldMapping, -} from './external-lookup.zod'; - -describe('ExternalDataSourceSchema', () => { - it('should validate complete external data source', () => { - const validSource: ExternalDataSource = { - id: 'salesforce-accounts', - name: 'Salesforce Account Data', - type: 'rest-api', - endpoint: 'https://api.salesforce.com/services/data/v58.0', - authentication: { - type: 'oauth2', - config: { - clientId: 'client_123', - clientSecret: 'secret_456', - tokenUrl: 'https://login.salesforce.com/services/oauth2/token', - }, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(validSource)).not.toThrow(); - }); - - it('should accept all data source types', () => { - const types = ['odata', 'rest-api', 'graphql', 'custom'] as const; - - types.forEach((type) => { - const source = { - id: `source-${type}`, - name: `${type} Source`, - type, - endpoint: 'https://api.example.com', - authentication: { - type: 'none' as const, - config: {}, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(source)).not.toThrow(); - }); - }); - - it('should accept all authentication types', () => { - const authTypes = ['oauth2', 'api-key', 'basic', 'none'] as const; - - authTypes.forEach((authType) => { - const source = { - id: `auth-${authType}`, - name: 'Test Source', - type: 'rest-api' as const, - endpoint: 'https://api.example.com', - authentication: { - type: authType, - config: {}, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(source)).not.toThrow(); - }); - }); - - it('should validate API key authentication', () => { - const source = { - id: 'api-key-source', - name: 'API Key Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'api-key', - config: { - apiKey: 'sk-1234567890', - headerName: 'X-API-Key', - }, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(source)).not.toThrow(); - }); - - it('should validate basic authentication', () => { - const source = { - id: 'basic-auth-source', - name: 'Basic Auth Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'basic', - config: { - username: 'user', - password: 'pass', - }, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(source)).not.toThrow(); - }); - - it('should reject invalid endpoint URL', () => { - const invalidSource = { - id: 'invalid-source', - name: 'Invalid Source', - type: 'rest-api', - endpoint: 'not-a-url', - authentication: { - type: 'none', - config: {}, - }, - }; - - expect(() => ExternalDataSourceSchema.parse(invalidSource)).toThrow(); - }); -}); - -describe('ExternalFieldMappingSchema', () => { - it('should validate complete field mapping', () => { - const validMapping: ExternalFieldMapping = { - source: 'AccountName', - target: 'name', - type: 'text', - readonly: true, - }; - - expect(() => ExternalFieldMappingSchema.parse(validMapping)).not.toThrow(); - }); - - it('should accept minimal field mapping', () => { - const minimalMapping = { - source: 'ExternalField', - target: 'local_field', - type: 'text', - }; - - expect(() => ExternalFieldMappingSchema.parse(minimalMapping)).not.toThrow(); - }); - - it('should default readonly to true', () => { - const mapping = { - source: 'Field1', - target: 'field_1', - type: 'text', - }; - - const parsed = ExternalFieldMappingSchema.parse(mapping); - expect(parsed.readonly).toBe(true); - }); - - it('should accept writable field mapping', () => { - const writableMapping = { - source: 'Status', - target: 'status', - type: 'text', - readonly: false, - }; - - expect(() => ExternalFieldMappingSchema.parse(writableMapping)).not.toThrow(); - }); - - it('should accept various field types', () => { - const types = ['text', 'number', 'boolean', 'date', 'datetime', 'lookup']; - - types.forEach((type) => { - const mapping = { - source: 'Field', - target: 'field', - type, - }; - - expect(() => ExternalFieldMappingSchema.parse(mapping)).not.toThrow(); - }); - }); -}); - -describe('ExternalLookupSchema', () => { - it('should validate complete external lookup', () => { - const validLookup: ExternalLookup = { - fieldName: 'external_account', - dataSource: { - id: 'salesforce-api', - name: 'Salesforce', - type: 'rest-api', - endpoint: 'https://api.salesforce.com/services/data/v58.0', - authentication: { - type: 'oauth2', - config: { clientId: 'client_123' }, - }, - }, - query: { - endpoint: '/sobjects/Account', - method: 'GET', - parameters: { limit: 100 }, - }, - fieldMappings: [ - { - source: 'Name', - target: 'account_name', - type: 'text', - readonly: true, - }, - { - source: 'Industry', - target: 'industry', - type: 'text', - readonly: true, - }, - ], - caching: { - enabled: true, - ttl: 300, - strategy: 'ttl', - }, - fallback: { - enabled: true, - showError: true, - }, - rateLimit: { - requestsPerSecond: 10, - burstSize: 20, - }, - }; - - expect(() => ExternalLookupSchema.parse(validLookup)).not.toThrow(); - }); - - it('should accept minimal external lookup', () => { - const minimalLookup = { - fieldName: 'external_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [ - { - source: 'Field1', - target: 'field_1', - type: 'text', - }, - ], - }; - - expect(() => ExternalLookupSchema.parse(minimalLookup)).not.toThrow(); - }); - - it('should default query method to GET', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - }; - - const parsed = ExternalLookupSchema.parse(lookup); - expect(parsed.query.method).toBe('GET'); - }); - - it('should accept POST query method', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/search', - method: 'POST' as const, - parameters: { - query: 'search term', - }, - }, - fieldMappings: [], - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - - it('should default caching to enabled with 300s TTL', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - caching: {}, - }; - - const parsed = ExternalLookupSchema.parse(lookup); - expect(parsed.caching?.enabled).toBe(true); - expect(parsed.caching?.ttl).toBe(300); - expect(parsed.caching?.strategy).toBe('ttl'); - }); - - it('should accept all cache strategies', () => { - const strategies = ['lru', 'lfu', 'ttl'] as const; - - strategies.forEach((strategy) => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - caching: { - strategy, - }, - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - }); - - it('should validate custom cache TTL', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - caching: { - enabled: true, - ttl: 600, - strategy: 'ttl' as const, - }, - }; - - const parsed = ExternalLookupSchema.parse(lookup); - expect(parsed.caching?.ttl).toBe(600); - }); - - it('should default fallback to enabled with showError true', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - fallback: {}, - }; - - const parsed = ExternalLookupSchema.parse(lookup); - expect(parsed.fallback?.enabled).toBe(true); - expect(parsed.fallback?.showError).toBe(true); - }); - - it('should accept custom fallback value', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - fallback: { - enabled: true, - defaultValue: 'N/A', - showError: false, - }, - }; - - const parsed = ExternalLookupSchema.parse(lookup); - expect(parsed.fallback?.defaultValue).toBe('N/A'); - expect(parsed.fallback?.showError).toBe(false); - }); - - it('should validate rate limiting', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - rateLimit: { - requestsPerSecond: 5, - burstSize: 10, - }, - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - - it('should accept rate limit without burst size', () => { - const lookup = { - fieldName: 'test_field', - dataSource: { - id: 'source-1', - name: 'Source', - type: 'rest-api', - endpoint: 'https://api.example.com', - authentication: { - type: 'none', - config: {}, - }, - }, - query: { - endpoint: '/data', - }, - fieldMappings: [], - rateLimit: { - requestsPerSecond: 10, - }, - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - - it('should validate OData external lookup', () => { - const odataLookup = { - fieldName: 'odata_products', - dataSource: { - id: 'odata-service', - name: 'OData Product Service', - type: 'odata' as const, - endpoint: 'https://services.odata.org/V4/Northwind/Northwind.svc', - authentication: { - type: 'none' as const, - config: {}, - }, - }, - query: { - endpoint: '/Products', - method: 'GET' as const, - parameters: { - $filter: "ProductName eq 'Chai'", - $select: 'ProductID,ProductName,UnitPrice', - }, - }, - fieldMappings: [ - { - source: 'ProductID', - target: 'product_id', - type: 'number', - readonly: true, - }, - { - source: 'ProductName', - target: 'product_name', - type: 'text', - readonly: true, - }, - { - source: 'UnitPrice', - target: 'unit_price', - type: 'currency', - readonly: true, - }, - ], - }; - - expect(() => ExternalLookupSchema.parse(odataLookup)).not.toThrow(); - }); - - it('should validate GraphQL external lookup', () => { - const graphqlLookup = { - fieldName: 'graphql_users', - dataSource: { - id: 'graphql-api', - name: 'GraphQL API', - type: 'graphql' as const, - endpoint: 'https://api.example.com/graphql', - authentication: { - type: 'api-key' as const, - config: { - apiKey: 'key_123', - headerName: 'Authorization', - }, - }, - }, - query: { - endpoint: '', - method: 'POST' as const, - parameters: { - query: '{ users { id name email } }', - }, - }, - fieldMappings: [ - { - source: 'id', - target: 'user_id', - type: 'text', - readonly: true, - }, - { - source: 'name', - target: 'user_name', - type: 'text', - readonly: true, - }, - { - source: 'email', - target: 'user_email', - type: 'email', - readonly: true, - }, - ], - caching: { - enabled: true, - ttl: 180, - strategy: 'lru' as const, - }, - }; - - expect(() => ExternalLookupSchema.parse(graphqlLookup)).not.toThrow(); - }); - - it('should validate complete Salesforce-like external lookup', () => { - const salesforceLookup: ExternalLookup = { - fieldName: 'salesforce_contacts', - dataSource: { - id: 'salesforce-prod', - name: 'Salesforce Production', - type: 'rest-api', - endpoint: 'https://na1.salesforce.com/services/data/v58.0', - authentication: { - type: 'oauth2', - config: { - clientId: 'client_id', - clientSecret: 'client_secret', - tokenUrl: 'https://login.salesforce.com/services/oauth2/token', - scope: 'api', - }, - }, - }, - query: { - endpoint: '/query', - method: 'GET', - parameters: { - q: 'SELECT Id, Name, Email, Phone FROM Contact WHERE IsActive = true LIMIT 1000', - }, - }, - fieldMappings: [ - { - source: 'Id', - target: 'salesforce_id', - type: 'text', - readonly: true, - }, - { - source: 'Name', - target: 'contact_name', - type: 'text', - readonly: true, - }, - { - source: 'Email', - target: 'email', - type: 'email', - readonly: true, - }, - { - source: 'Phone', - target: 'phone', - type: 'phone', - readonly: true, - }, - ], - caching: { - enabled: true, - ttl: 600, - strategy: 'ttl', - }, - fallback: { - enabled: true, - defaultValue: null, - showError: true, - }, - rateLimit: { - requestsPerSecond: 5, - burstSize: 15, - }, - }; - - expect(() => ExternalLookupSchema.parse(salesforceLookup)).not.toThrow(); - }); -}); - -describe('External Lookup Retry Configuration', () => { - const baseLookup = { - fieldName: 'external_data', - dataSource: { - id: 'test-api', - name: 'Test API', - type: 'rest-api' as const, - endpoint: 'https://api.example.com', - authentication: { type: 'api-key' as const, config: { key: 'test' } }, - }, - query: { endpoint: '/data' }, - fieldMappings: [{ source: 'name', target: 'name' }], - }; - - it('should accept lookup with retry configuration', () => { - const lookup = { - ...baseLookup, - retry: { - maxRetries: 3, - initialDelayMs: 1000, - maxDelayMs: 30000, - backoffMultiplier: 2, - retryableStatusCodes: [429, 500, 502, 503, 504], - }, - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - - it('should apply retry defaults', () => { - const result = ExternalLookupSchema.parse({ - ...baseLookup, - retry: {}, - }); - - expect(result.retry?.maxRetries).toBe(3); - expect(result.retry?.initialDelayMs).toBe(1000); - expect(result.retry?.maxDelayMs).toBe(30000); - expect(result.retry?.backoffMultiplier).toBe(2); - expect(result.retry?.retryableStatusCodes).toEqual([429, 500, 502, 503, 504]); - }); - - it('should accept custom retryable status codes', () => { - const result = ExternalLookupSchema.parse({ - ...baseLookup, - retry: { retryableStatusCodes: [408, 429, 503] }, - }); - - expect(result.retry?.retryableStatusCodes).toEqual([408, 429, 503]); - }); - - it('should reject negative maxRetries', () => { - expect(() => ExternalLookupSchema.parse({ - ...baseLookup, - retry: { maxRetries: -1 }, - })).toThrow(); - }); -}); - -describe('External Lookup Transform Configuration', () => { - const baseLookup = { - fieldName: 'external_data', - dataSource: { - id: 'test-api', - name: 'Test API', - type: 'rest-api' as const, - endpoint: 'https://api.example.com', - authentication: { type: 'api-key' as const, config: { key: 'test' } }, - }, - query: { endpoint: '/data' }, - fieldMappings: [{ source: 'name', target: 'name' }], - }; - - it('should accept lookup with transform pipeline', () => { - const lookup = { - ...baseLookup, - transform: { - request: { - headers: { 'X-Custom-Header': 'value' }, - queryParams: { format: 'json' }, - }, - response: { - dataPath: '$.data.results', - totalPath: '$.meta.total', - }, - }, - }; - - expect(() => ExternalLookupSchema.parse(lookup)).not.toThrow(); - }); - - it('should accept partial transform config', () => { - const result = ExternalLookupSchema.parse({ - ...baseLookup, - transform: { - response: { dataPath: '$.items' }, - }, - }); - - expect(result.transform?.response?.dataPath).toBe('$.items'); - expect(result.transform?.request).toBeUndefined(); - }); -}); - -describe('External Lookup Pagination Configuration', () => { - const baseLookup = { - fieldName: 'external_data', - dataSource: { - id: 'test-api', - name: 'Test API', - type: 'rest-api' as const, - endpoint: 'https://api.example.com', - authentication: { type: 'api-key' as const, config: { key: 'test' } }, - }, - query: { endpoint: '/data' }, - fieldMappings: [{ source: 'name', target: 'name' }], - }; - - it('should accept lookup with pagination', () => { - const result = ExternalLookupSchema.parse({ - ...baseLookup, - pagination: { - type: 'cursor', - pageSize: 50, - maxPages: 10, - }, - }); - - expect(result.pagination?.type).toBe('cursor'); - expect(result.pagination?.pageSize).toBe(50); - }); - - it('should apply pagination defaults', () => { - const result = ExternalLookupSchema.parse({ - ...baseLookup, - pagination: {}, - }); - - expect(result.pagination?.type).toBe('offset'); - expect(result.pagination?.pageSize).toBe(100); - }); - - it('should accept all pagination types', () => { - const types = ['offset', 'cursor', 'page'] as const; - types.forEach(type => { - expect(() => ExternalLookupSchema.parse({ - ...baseLookup, - pagination: { type }, - })).not.toThrow(); - }); - }); -}); diff --git a/packages/spec/src/data/external-lookup.zod.ts b/packages/spec/src/data/external-lookup.zod.ts deleted file mode 100644 index 734f59193e..0000000000 --- a/packages/spec/src/data/external-lookup.zod.ts +++ /dev/null @@ -1,315 +0,0 @@ -// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. - -import { z } from 'zod'; -import { FieldMappingSchema as BaseFieldMappingSchema } from '../shared/mapping.zod'; - -/** - * External Data Source Schema - * - * Configuration for connecting to external data systems. - * Similar to Salesforce External Objects for real-time data integration. - * - * @example - * ```json - * { - * "id": "salesforce-accounts", - * "name": "Salesforce Account Data", - * "type": "rest-api", - * "endpoint": "https://api.salesforce.com/services/data/v58.0", - * "authentication": { - * "type": "oauth2", - * "config": { - * "clientId": "...", - * "clientSecret": "...", - * "tokenUrl": "https://login.salesforce.com/services/oauth2/token" - * } - * } - * } - * ``` - */ -import { lazySchema } from '../shared/lazy-schema'; -export const ExternalDataSourceSchema = lazySchema(() => z.object({ - /** - * Unique identifier for the external data source - */ - id: z.string().describe('Data source ID'), - - /** - * Human-readable name of the data source - */ - name: z.string().describe('Data source name'), - - /** - * Protocol type for connecting to the data source - */ - type: z.enum(['odata', 'rest-api', 'graphql', 'custom']).describe('Protocol type'), - - /** - * Base URL endpoint for the external system - */ - endpoint: z.string().url().describe('API endpoint URL'), - - /** - * Authentication configuration - */ - authentication: z.object({ - /** - * Authentication method - */ - type: z.enum(['oauth2', 'api-key', 'basic', 'none']).describe('Auth type'), - - /** - * Authentication-specific configuration - * Structure varies based on auth type - */ - config: z.record(z.string(), z.unknown()).describe('Auth configuration'), - }).describe('Authentication'), -})); - -/** - * Field Mapping Schema for External Lookups - * - * Extends the base field mapping with external lookup specific features. - * Uses the canonical field mapping protocol from shared/mapping.zod.ts. - * - * @see {@link BaseFieldMappingSchema} for the base field mapping schema - * - * @example - * ```json - * { - * "source": "AccountName", - * "target": "name", - * "readonly": true - * } - * ``` - */ -export const ExternalFieldMappingSchema = lazySchema(() => BaseFieldMappingSchema.extend({ - /** - * Field data type - */ - type: z.string().optional().describe('Field type'), - - /** - * Whether the field is read-only - * @default true - */ - readonly: z.boolean().optional().default(true).describe('Read-only field'), -})); - -/** - * External Lookup Schema - * - * Real-time data lookup protocol for external systems. - * Enables querying external data sources without replication. - * Inspired by Salesforce External Objects and OData protocols. - * - * @example - * ```json - * { - * "fieldName": "external_account", - * "dataSource": { - * "id": "salesforce-api", - * "name": "Salesforce", - * "type": "rest-api", - * "endpoint": "https://api.salesforce.com/services/data/v58.0", - * "authentication": { - * "type": "oauth2", - * "config": {"clientId": "..."} - * } - * }, - * "query": { - * "endpoint": "/sobjects/Account", - * "method": "GET", - * "parameters": {"limit": 100} - * }, - * "fieldMappings": [ - * { - * "externalField": "Name", - * "localField": "account_name", - * "type": "text", - * "readonly": true - * } - * ], - * "caching": { - * "enabled": true, - * "ttl": 300, - * "strategy": "ttl" - * }, - * "fallback": { - * "enabled": true, - * "showError": true - * }, - * "rateLimit": { - * "requestsPerSecond": 10, - * "burstSize": 20 - * } - * } - * ``` - */ -export const ExternalLookupSchema = lazySchema(() => z.object({ - /** - * Name of the field that uses external lookup - */ - fieldName: z.string().describe('Field name'), - - /** - * External data source configuration - */ - dataSource: ExternalDataSourceSchema.describe('External data source'), - - /** - * Query configuration for fetching external data - */ - query: z.object({ - /** - * API endpoint path (relative to base endpoint) - */ - endpoint: z.string().describe('Query endpoint path'), - - /** - * HTTP method for the query - * @default 'GET' - */ - method: z.enum(['GET', 'POST']).optional().default('GET').describe('HTTP method'), - - /** - * Query parameters or request body - */ - parameters: z.record(z.string(), z.unknown()).optional().describe('Query parameters'), - }).describe('Query configuration'), - - /** - * Mapping between external and local fields - */ - fieldMappings: z.array(ExternalFieldMappingSchema).describe('Field mappings'), - - /** - * Cache configuration for external data - */ - caching: z.object({ - /** - * Whether caching is enabled - * @default true - */ - enabled: z.boolean().optional().default(true).describe('Cache enabled'), - - /** - * Time-to-live in seconds - * @default 300 - */ - ttl: z.number().optional().default(300).describe('Cache TTL (seconds)'), - - /** - * Cache eviction strategy - * @default 'ttl' - */ - strategy: z.enum(['lru', 'lfu', 'ttl']).optional().default('ttl').describe('Cache strategy'), - }).optional().describe('Caching configuration'), - - /** - * Fallback behavior when external system is unavailable - */ - fallback: z.object({ - /** - * Whether fallback is enabled - * @default true - */ - enabled: z.boolean().optional().default(true).describe('Fallback enabled'), - - /** - * Default value to use when external system fails - */ - defaultValue: z.unknown().optional().describe('Default fallback value'), - - /** - * Whether to show error message to user - * @default true - */ - showError: z.boolean().optional().default(true).describe('Show error to user'), - }).optional().describe('Fallback configuration'), - - /** - * Rate limiting to prevent overwhelming external system - */ - rateLimit: z.object({ - /** - * Maximum requests per second - */ - requestsPerSecond: z.number().describe('Requests per second limit'), - - /** - * Burst size for handling spikes - */ - burstSize: z.number().optional().describe('Burst size'), - }).optional().describe('Rate limiting'), - - /** - * Retry configuration with exponential backoff - * - * @example - * ```json - * { - * "maxRetries": 3, - * "initialDelayMs": 1000, - * "maxDelayMs": 30000, - * "backoffMultiplier": 2, - * "retryableStatusCodes": [429, 500, 502, 503, 504] - * } - * ``` - */ - retry: z.object({ - /** Maximum number of retry attempts */ - maxRetries: z.number().min(0).default(3).describe('Maximum retry attempts'), - /** Initial delay before first retry (ms) */ - initialDelayMs: z.number().default(1000).describe('Initial retry delay in milliseconds'), - /** Maximum delay between retries (ms) */ - maxDelayMs: z.number().default(30000).describe('Maximum retry delay in milliseconds'), - /** Backoff multiplier for exponential backoff */ - backoffMultiplier: z.number().default(2).describe('Exponential backoff multiplier'), - /** HTTP status codes that trigger a retry */ - retryableStatusCodes: z.array(z.number()).default([429, 500, 502, 503, 504]) - .describe('HTTP status codes that are retryable'), - }).optional().describe('Retry configuration with exponential backoff'), - - /** - * Request/response transformation pipeline - * - * Allows transforming request parameters and response data - * before they are processed by the external lookup system. - */ - transform: z.object({ - /** Transform request parameters before sending */ - request: z.object({ - /** Header transformations (key-value additions) */ - headers: z.record(z.string(), z.string()).optional().describe('Additional request headers'), - /** Query parameter transformations */ - queryParams: z.record(z.string(), z.string()).optional().describe('Additional query parameters'), - }).optional().describe('Request transformation'), - /** Transform response data after receiving */ - response: z.object({ - /** JSONPath expression to extract data from response */ - dataPath: z.string().optional().describe('JSONPath to extract data (e.g., "$.data.results")'), - /** JSONPath expression to extract total count for pagination */ - totalPath: z.string().optional().describe('JSONPath to extract total count (e.g., "$.meta.total")'), - }).optional().describe('Response transformation'), - }).optional().describe('Request/response transformation pipeline'), - - /** Pagination support for external data sources */ - pagination: z.object({ - /** Pagination type */ - type: z.enum(['offset', 'cursor', 'page']).default('offset').describe('Pagination type'), - /** Page size */ - pageSize: z.number().default(100).describe('Items per page'), - /** Maximum pages to fetch */ - maxPages: z.number().optional().describe('Maximum number of pages to fetch'), - }).optional().describe('Pagination configuration for external data'), -})); - -// Type exports -export type ExternalLookup = z.input; -/** Post-parse shape of {@link ExternalLookup} — defaults applied, transforms run (ADR-0122). */ -export type ExternalLookupParsed = z.infer; -export type ExternalDataSource = z.input; -export type ExternalFieldMapping = z.input; -/** Post-parse shape of {@link ExternalFieldMapping} — defaults applied, transforms run (ADR-0122). */ -export type ExternalFieldMappingParsed = z.infer; diff --git a/packages/spec/src/data/index.ts b/packages/spec/src/data/index.ts index b677027040..36c8792d6e 100644 --- a/packages/spec/src/data/index.ts +++ b/packages/spec/src/data/index.ts @@ -105,8 +105,27 @@ export * from './seed-loader.zod'; // Document Management Protocol export * from './document.zod'; -// External Lookup Protocol -export * from './external-lookup.zod'; +// external-lookup.zod (ExternalDataSourceSchema / ExternalFieldMappingSchema / +// ExternalLookupSchema + every type alias) was REMOVED per ADR-0049 +// enforce-or-remove (#8075). The module declared a real-time external-data +// lookup protocol — a per-field external data source with an +// `authentication.config` record whose own docblock example wrote an inline +// `clientSecret` — and nothing anywhere consumed it: no metadata-type binding, +// no stack collection, no object/field embedding (`object.external` binds +// `ObjectExternalBindingSchema`, which carries no credentials), and zero +// imports outside this package in objectstack. An exported schema with no +// consumer reads as a capability to whoever finds it (#3950) — here it read as +// an invitation to put OAuth client secrets in cleartext metadata. +// +// The live mechanism external data actually goes through: `object.external` +// (`ObjectExternalBindingSchema` in object.zod.ts, ADR-0015/0062) names a +// datasource by reference, and connection credentials live in the datasource +// config (`datasource.zod.ts` / `driver/`), never inline in object metadata. +// `external-catalog.zod.ts` below is that federated path's catalog surface and +// is NOT part of this retirement. Real-time per-field lookup, if ever built, +// returns via the enforce route of ADR-0049 through a new ADR — the executor +// first, the vocabulary second. See the D3 record +// `external-lookup-message-queue-families-retired`. export * from './datasource.zod'; // Per-driver `datasource.config` contracts (#4410) — the enforcement half of diff --git a/packages/spec/src/integration/connector.test.ts b/packages/spec/src/integration/connector.test.ts index 7daef24fae..c1a67f84bc 100644 --- a/packages/spec/src/integration/connector.test.ts +++ b/packages/spec/src/integration/connector.test.ts @@ -907,13 +907,14 @@ describe('[#4911] `./integration` no longer publishes an outbound rate-limit sha // CONCEPT: the column mapping of a CSV/table import. // // The first two are base-and-superset, so "converge them" is a tempting read. -// It is wrong in both directions: widening the base to 7 keys pushes connector -// sync semantics onto `data/external-lookup.zod.ts` which also extends it -// (`automation/sync.zod.ts` embedded the base too until its retirement in -// #4738), and narrowing the connector side to 4 is a retirement of +// It is wrong in both directions: widening the base to 7 keys pushed connector +// sync semantics onto every other extender of the base +// (`data/external-lookup.zod.ts` extended it until its #8075 retirement; +// `automation/sync.zod.ts` embedded it too until its retirement in #4738), and +// narrowing the connector side to 4 is a retirement of // three live keys, not a naming fix. ADR-0112 D9a's prefix remedy applies, and -// the file next door already demonstrates it: `data/ExternalFieldMappingSchema` -// extends the same base and, purely because it carries a prefix, never entered +// the file next door already demonstrated it: `data/ExternalFieldMappingSchema` +// extended the same base and, purely because it carried a prefix, never entered // the dual-source baseline at all. // // The `./data` side is not a spelling variant of anything. The tests below pin diff --git a/packages/spec/src/integration/connector.zod.ts b/packages/spec/src/integration/connector.zod.ts index 8543de3930..e11744a862 100644 --- a/packages/spec/src/integration/connector.zod.ts +++ b/packages/spec/src/integration/connector.zod.ts @@ -153,9 +153,10 @@ import { retiredKey } from '../shared/retired-key'; * base name for the base, matching `ConnectorErrorCategory` and * `ConnectorRetryStrategy` in this same file (`ConnectorRateLimitConfig`, * #4684, was the fourth until its whole shape was retired in #4911), and - * `ExternalFieldMappingSchema` in `data/external-lookup.zod.ts` — which extends - * the same base and, precisely because it carries a domain prefix, never - * entered the dual-source baseline. + * `ExternalFieldMappingSchema` in `data/external-lookup.zod.ts` — which + * extended the same base and, precisely because it carried a domain prefix, + * never entered the dual-source baseline (the external-lookup family was + * itself retired whole in #8075 — ADR-0049, zero consumers). */ import { lazySchema } from '../shared/lazy-schema'; export const ConnectorFieldMappingSchema = lazySchema(() => BaseFieldMappingSchema.extend({ diff --git a/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalDataSource.ts b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalDataSource.ts new file mode 100644 index 0000000000..22031d64f5 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalDataSource.ts @@ -0,0 +1,14 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049 +// enforce-or-remove; fork (b) of the #8075 census, accepted 2026-08-12). The +// external-data-source shape whose `authentication.config` record accepted +// inline OAuth client secrets and API keys — its own docblock example wrote +// `"clientSecret": "..."`. Zero consumers: no metadata-type binding, no stack +// collection, no object/field embedding (`object.external` binds +// `ObjectExternalBindingSchema`, which routes credentials through datasource +// config per ADR-0015/0062), and its only in-module consumer +// (`ExternalLookupSchema.dataSource`) was itself consumed by nothing. Route 3: +// no tombstone, no D2 conversion — this table plus the D3 +// `external-lookup-message-queue-families-retired` are the declaration. +export const entry = 'data/ExternalDataSource'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalFieldMapping.ts b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalFieldMapping.ts new file mode 100644 index 0000000000..6db5a64089 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalFieldMapping.ts @@ -0,0 +1,12 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049). Extended +// `shared/FieldMapping` and was embedded only by `ExternalLookupSchema`, which +// nothing consumed. Its `transform` key's #5552 `retiredKey()` tombstone — and +// the `data/ExternalFieldMapping:transform` RETIRED_KEYS entry that registered +// it — are SUBSUMED here, the WidgetManifest.performance way: they go with the +// shape that carried them, which is strictly stronger, because there is no +// longer a mapping shape to author the key INTO. The base tombstone on +// `shared/FieldMapping` and the `integration/ConnectorFieldMapping` spelling +// are untouched and still reject the key with the #5552 prescription. +export const entry = 'data/ExternalFieldMapping'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalLookup.ts b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalLookup.ts new file mode 100644 index 0000000000..6b2c74e8e1 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.data__ExternalLookup.ts @@ -0,0 +1,11 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049). The root of +// the module's dependency shape: it embedded `ExternalDataSource` (the inline +// credential sink) and `ExternalFieldMapping`, and was itself referenced by no +// stack collection, metadata type or import outside packages/spec — the +// #5552 conversion's docblock had already recorded that no external-lookup +// document exists for the conversion walker to visit. Real-time external +// lookup returns via the enforce route of ADR-0049: the executor first, the +// vocabulary second. +export const entry = 'data/ExternalLookup'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.system__ConsumerConfig.ts b/packages/spec/src/migrations/entries/retired-defs/17.system__ConsumerConfig.ts new file mode 100644 index 0000000000..f0ca17f799 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.system__ConsumerConfig.ts @@ -0,0 +1,7 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). Consumer +// group config (groupId / offset reset / auto-commit / poll size), embedded +// only by `system/MessageQueueConfig` (retired in the same change) and +// consumed by nothing. +export const entry = 'system/ConsumerConfig'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.system__DeadLetterQueue.ts b/packages/spec/src/migrations/entries/retired-defs/17.system__DeadLetterQueue.ts new file mode 100644 index 0000000000..1c4c4b9473 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.system__DeadLetterQueue.ts @@ -0,0 +1,9 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). DLQ config, +// embedded only by `system/MessageQueueConfig` (retired in the same change) +// and consumed by nothing. ⚠️ `kernel/DeadLetterQueueEntry` is a different +// declaration (the event bus's per-event DLQ record) and is untouched — +// name adjacency is not evidence, the `system/ServerRateLimitConfig` note +// applies verbatim. +export const entry = 'system/DeadLetterQueue'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueConfig.ts b/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueConfig.ts new file mode 100644 index 0000000000..6e428a2ace --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueConfig.ts @@ -0,0 +1,14 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — system/message-queue.zod.ts, retired whole (ADR-0049 +// enforce-or-remove; fork (b) of the #8075 census, accepted 2026-08-12). The +// top-level broker-administration config, carrying a required inline +// `sasl.password` credential when `sasl` was present. Zero consumers outside +// packages/spec repo-wide; no `message_queue` metadata type; the connector +// 'message_queue' ConnectorType enum value never referenced this schema. The +// consumed near-namesake `kernel/EventMessageQueueConfig` +// (`EventBusConfigSchema.messageQueue`) deliberately carries NO credential +// field and is untouched. Route 3: no tombstone, no D2 conversion — this +// table plus the D3 `external-lookup-message-queue-families-retired` are the +// declaration. +export const entry = 'system/MessageQueueConfig'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueProvider.ts b/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueProvider.ts new file mode 100644 index 0000000000..36736ba159 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.system__MessageQueueProvider.ts @@ -0,0 +1,9 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). The +// six-value broker provider enum, embedded only by +// `system/MessageQueueConfig` (retired in the same change). Not the same +// declaration as `kernel/EventMessageQueueConfig.provider` — that is its own +// INLINE `z.enum` (same six values, never a reference to this def) and is +// untouched. +export const entry = 'system/MessageQueueProvider'; diff --git a/packages/spec/src/migrations/entries/retired-defs/17.system__TopicConfig.ts b/packages/spec/src/migrations/entries/retired-defs/17.system__TopicConfig.ts new file mode 100644 index 0000000000..42e88b4fe0 --- /dev/null +++ b/packages/spec/src/migrations/entries/retired-defs/17.system__TopicConfig.ts @@ -0,0 +1,8 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +// #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). Topic +// partitions/replication/retention/compression config, embedded only by +// `system/MessageQueueConfig` (retired in the same change) and consumed by +// nothing — leaving it behind would strand an exported schema with no +// consumer (#3950). +export const entry = 'system/TopicConfig'; diff --git a/packages/spec/src/migrations/entries/retired-keys/17.data__ExternalFieldMapping__transform.ts b/packages/spec/src/migrations/entries/retired-keys/17.data__ExternalFieldMapping__transform.ts deleted file mode 100644 index 0ce8abcc57..0000000000 --- a/packages/spec/src/migrations/entries/retired-keys/17.data__ExternalFieldMapping__transform.ts +++ /dev/null @@ -1,3 +0,0 @@ -// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. - -export const entry = 'data/ExternalFieldMapping:transform'; diff --git a/packages/spec/src/migrations/entries/semantic/17.external-lookup-message-queue-families-retired.ts b/packages/spec/src/migrations/entries/semantic/17.external-lookup-message-queue-families-retired.ts new file mode 100644 index 0000000000..58b0058a3b --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/17.external-lookup-message-queue-families-retired.ts @@ -0,0 +1,73 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +export const entry: SemanticMigration = { + id: 'external-lookup-message-queue-families-retired', + surface: + 'data.externalLookup / data.externalDataSource / data.externalFieldMapping ' + + '(the whole of data/external-lookup.zod.ts — 3 defs, 8 exported names) and ' + + 'system.messageQueue (the whole of system/message-queue.zod.ts — ' + + 'MessageQueueConfig, MessageQueueProvider, TopicConfig, ConsumerConfig, ' + + 'DeadLetterQueue — 5 defs, 14 exported names)', + replacement: + '(removed — there is no replacement key, because there was never a key: neither ' + + 'family was reachable from any metadata-type binding, stack collection or /meta ' + + 'door, so no document could carry either. For external data: `object.external` ' + + '(`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference ' + + 'and connection credentials live in the datasource config — never inline in ' + + 'object metadata; `data/external-catalog.zod.ts` is that federated path\'s ' + + 'catalog surface and is untouched. For message queues: the LIVE surface is ' + + '`kernel/events/integrations.zod.ts`\'s `EventMessageQueueConfig` ' + + '(`EventBusConfig.messageQueue`), which deliberately carries NO credential ' + + 'field — broker connection and SASL credentials are runtime deployment ' + + 'configuration, not authorable metadata. Either capability returns via the ' + + 'ENFORCE route of ADR-0049 through a new ADR — the executor / broker admin ' + + 'service first, the vocabulary second)', + reason: + 'Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): ' + + 'security-shaped declared surface with inline-credential sinks and ZERO ' + + 'consumers. `ExternalDataSourceSchema.authentication.config` is a record of ' + + 'unknown whose own docblock example wrote `"clientSecret": "..."` inline, and ' + + '`MessageQueueConfigSchema.sasl.password` was a required inline broker ' + + 'credential — the #7990 class (cleartext-at-rest credential sinks), except ' + + 'that unlike #7990\'s two measured surfaces nothing ever persisted these: no ' + + 'metadata-type binding (kernel/metadata-type-schemas.ts imports neither ' + + 'module), no stack collection, no object/field embedding (`object.external` ' + + 'binds `ObjectExternalBindingSchema` — remoteName/remoteSchema/writable/' + + 'columnMap, no authentication), and zero imports outside packages/spec ' + + 'repo-wide, with the corpus-reach control (`DatasourceSchema` under identical ' + + 'exclusions) returning hits in the same run. The consumed MQ near-namesake ' + + '`kernel/EventMessageQueueConfig` deliberately has no credential key, so the ' + + 'consumed shape had no credential and the credential-bearing shape had no ' + + 'consumer. A dead schema minus one field is still a dead schema, so the whole ' + + 'declarations go, not just the credential faces (#3950: an exported schema ' + + 'with no consumer reads as a capability to whoever finds it — here it read as ' + + 'an invitation to author secrets in cleartext). With no carrier key there is ' + + 'nothing to tombstone and no source or `sys_metadata` row for a D2 conversion ' + + 'to rewrite: route 3, the #4834 / #4988 / #5055 / #6486 shape — ' + + 'RETIRED_DEFS_BY_MAJOR plus this entry ARE the declaration. ' + + '⚠️ The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that ' + + 'retirement\'s three spellings) is SUBSUMED by the def retirement, the ' + + 'WidgetManifest.performance way: it goes with the shape that carried it. The ' + + 'base `shared/FieldMapping` tombstone and the `integration/' + + 'ConnectorFieldMapping` spelling are untouched and still reject `transform` ' + + 'with the #5552 prescription. ' + + '⚠️ The #7990 Option-B reopen trigger ("a third measured artefact-type ' + + 'surface") is NOT met by this census — that ruling\'s parked class-level ' + + 'write-boundary guard stays parked; this is the ADR-0049 leg of the fork the ' + + 'triage pre-agreed.', + acceptanceCriteria: + 'No code imports `ExternalLookup(Schema|Parsed)`, `ExternalDataSource(Schema)`, ' + + '`ExternalFieldMapping(Schema|Parsed)`, `MessageQueueConfig(Schema|Parsed)`, ' + + '`MessageQueueProvider(Schema)`, `TopicConfig(Schema|Parsed)`, ' + + '`ConsumerConfig(Schema|Parsed)` or `DeadLetterQueue(Schema|Parsed)` from ' + + '`@objectstack/spec`, `@objectstack/spec/data` or `@objectstack/spec/system` — ' + + 'every one is TS2305 after upgrade, on every public entry (pinned by resolved ' + + 'symbol identity in `data/external-lookup-retirement.test.ts` and ' + + '`system/message-queue-retirement.test.ts`). No metadata document needs ' + + 'editing, because none could ever carry one of these shapes. ' + + '`kernel/EventMessageQueueConfig` (with its inline provider enum and no ' + + 'credential key), `data/external-catalog.zod.ts`, `object.external` and ' + + '`kernel/DeadLetterQueueEntry` survive unchanged.', +}; diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 2426819c63..536e3024a1 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -2762,6 +2762,75 @@ const step17: MigrationStep = { + "engine's `applyFieldDefaults` gate, so read `fields[name].defaultValue` and apply " + 'that same `!= null` test yourself.', }, + { + id: 'external-lookup-message-queue-families-retired', + surface: + 'data.externalLookup / data.externalDataSource / data.externalFieldMapping ' + + '(the whole of data/external-lookup.zod.ts — 3 defs, 8 exported names) and ' + + 'system.messageQueue (the whole of system/message-queue.zod.ts — ' + + 'MessageQueueConfig, MessageQueueProvider, TopicConfig, ConsumerConfig, ' + + 'DeadLetterQueue — 5 defs, 14 exported names)', + replacement: + '(removed — there is no replacement key, because there was never a key: neither ' + + 'family was reachable from any metadata-type binding, stack collection or /meta ' + + 'door, so no document could carry either. For external data: `object.external` ' + + '(`ObjectExternalBindingSchema`, ADR-0015/0062) names a datasource by reference ' + + 'and connection credentials live in the datasource config — never inline in ' + + 'object metadata; `data/external-catalog.zod.ts` is that federated path\'s ' + + 'catalog surface and is untouched. For message queues: the LIVE surface is ' + + '`kernel/events/integrations.zod.ts`\'s `EventMessageQueueConfig` ' + + '(`EventBusConfig.messageQueue`), which deliberately carries NO credential ' + + 'field — broker connection and SASL credentials are runtime deployment ' + + 'configuration, not authorable metadata. Either capability returns via the ' + + 'ENFORCE route of ADR-0049 through a new ADR — the executor / broker admin ' + + 'service first, the vocabulary second)', + reason: + 'Both families are the #8075 census verdict (fork (b), accepted 2026-08-12): ' + + 'security-shaped declared surface with inline-credential sinks and ZERO ' + + 'consumers. `ExternalDataSourceSchema.authentication.config` is a record of ' + + 'unknown whose own docblock example wrote `"clientSecret": "..."` inline, and ' + + '`MessageQueueConfigSchema.sasl.password` was a required inline broker ' + + 'credential — the #7990 class (cleartext-at-rest credential sinks), except ' + + 'that unlike #7990\'s two measured surfaces nothing ever persisted these: no ' + + 'metadata-type binding (kernel/metadata-type-schemas.ts imports neither ' + + 'module), no stack collection, no object/field embedding (`object.external` ' + + 'binds `ObjectExternalBindingSchema` — remoteName/remoteSchema/writable/' + + 'columnMap, no authentication), and zero imports outside packages/spec ' + + 'repo-wide, with the corpus-reach control (`DatasourceSchema` under identical ' + + 'exclusions) returning hits in the same run. The consumed MQ near-namesake ' + + '`kernel/EventMessageQueueConfig` deliberately has no credential key, so the ' + + 'consumed shape had no credential and the credential-bearing shape had no ' + + 'consumer. A dead schema minus one field is still a dead schema, so the whole ' + + 'declarations go, not just the credential faces (#3950: an exported schema ' + + 'with no consumer reads as a capability to whoever finds it — here it read as ' + + 'an invitation to author secrets in cleartext). With no carrier key there is ' + + 'nothing to tombstone and no source or `sys_metadata` row for a D2 conversion ' + + 'to rewrite: route 3, the #4834 / #4988 / #5055 / #6486 shape — ' + + 'RETIRED_DEFS_BY_MAJOR plus this entry ARE the declaration. ' + + '⚠️ The #5552 `data/ExternalFieldMapping:transform` tombstone (one of that ' + + 'retirement\'s three spellings) is SUBSUMED by the def retirement, the ' + + 'WidgetManifest.performance way: it goes with the shape that carried it. The ' + + 'base `shared/FieldMapping` tombstone and the `integration/' + + 'ConnectorFieldMapping` spelling are untouched and still reject `transform` ' + + 'with the #5552 prescription. ' + + '⚠️ The #7990 Option-B reopen trigger ("a third measured artefact-type ' + + 'surface") is NOT met by this census — that ruling\'s parked class-level ' + + 'write-boundary guard stays parked; this is the ADR-0049 leg of the fork the ' + + 'triage pre-agreed.', + acceptanceCriteria: + 'No code imports `ExternalLookup(Schema|Parsed)`, `ExternalDataSource(Schema)`, ' + + '`ExternalFieldMapping(Schema|Parsed)`, `MessageQueueConfig(Schema|Parsed)`, ' + + '`MessageQueueProvider(Schema)`, `TopicConfig(Schema|Parsed)`, ' + + '`ConsumerConfig(Schema|Parsed)` or `DeadLetterQueue(Schema|Parsed)` from ' + + '`@objectstack/spec`, `@objectstack/spec/data` or `@objectstack/spec/system` — ' + + 'every one is TS2305 after upgrade, on every public entry (pinned by resolved ' + + 'symbol identity in `data/external-lookup-retirement.test.ts` and ' + + '`system/message-queue-retirement.test.ts`). No metadata document needs ' + + 'editing, because none could ever carry one of these shapes. ' + + '`kernel/EventMessageQueueConfig` (with its inline provider enum and no ' + + 'credential key), `data/external-catalog.zod.ts`, `object.external` and ' + + '`kernel/DeadLetterQueueEntry` survive unchanged.', + }, { id: 'filter-regex-options-retired', // No backticks in `surface` — build-upgrade-guide.ts renders it inside a @@ -4379,11 +4448,14 @@ export const MIGRATION_MAJORS: readonly number[] = Object.keys(MIGRATIONS_BY_MAJ */ export const RETIRED_KEYS_BY_MAJOR: Readonly> = { // The first entries since #4659 built this table (#5552). ONE tombstone - // produces THREE keys: `transform` is declared on `shared/FieldMapping` and - // `integration/ConnectorFieldMapping` / `data/ExternalFieldMapping` are - // `.extend()`s of it, so the retired property is copied into all three walked - // shapes and `authorable-surface.json` marks each `[RETIRED]` separately. - // Registered per key, as the gate reads them — nothing radiates from the base. + // produced THREE keys: `transform` is declared on `shared/FieldMapping`, and + // every `.extend()` of the base copies the retired property into its own + // walked shape, which `authorable-surface/` marks `[RETIRED]` separately. + // Registered per key, as the gate reads them — nothing radiates from the + // base. Two of the three remain: `data/ExternalFieldMapping:transform` left + // when #8075 retired the whole external-lookup family (the def retirement + // subsumes the key entry, the WidgetManifest.performance way — see + // RETIRED_DEFS_BY_MAJOR below). 17: [ // One file per entry under `entries/retired-keys/`, concatenated here sorted by // entry id by `gen:migration-registry` (#7297). Add an entry by adding a @@ -4468,7 +4540,6 @@ export const RETIRED_KEYS_BY_MAJOR: Readonly> // takes (`query-joins-retired` / `query-cursor-retired` / // `query-distinct-retired` / `query-window-functions-retired`, #4286). 'data/AggregationNode:distinct', - 'data/ExternalFieldMapping:transform', // #7990 — sibling of `data/PostgresConfig:password`, same ruling, same // disposition: tombstoned inline credential; the secret binder / // `external.credentialsRef` is the mechanism. See that entry for the reasoning @@ -4691,6 +4762,37 @@ export const RETIRED_DEFS_BY_MAJOR: Readonly> 'automation/ETLSyncMode', 'automation/ETLTransformation', 'automation/ETLTransformationType', + // #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049 + // enforce-or-remove; fork (b) of the #8075 census, accepted 2026-08-12). The + // external-data-source shape whose `authentication.config` record accepted + // inline OAuth client secrets and API keys — its own docblock example wrote + // `"clientSecret": "..."`. Zero consumers: no metadata-type binding, no stack + // collection, no object/field embedding (`object.external` binds + // `ObjectExternalBindingSchema`, which routes credentials through datasource + // config per ADR-0015/0062), and its only in-module consumer + // (`ExternalLookupSchema.dataSource`) was itself consumed by nothing. Route 3: + // no tombstone, no D2 conversion — this table plus the D3 + // `external-lookup-message-queue-families-retired` are the declaration. + 'data/ExternalDataSource', + // #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049). Extended + // `shared/FieldMapping` and was embedded only by `ExternalLookupSchema`, which + // nothing consumed. Its `transform` key's #5552 `retiredKey()` tombstone — and + // the `data/ExternalFieldMapping:transform` RETIRED_KEYS entry that registered + // it — are SUBSUMED here, the WidgetManifest.performance way: they go with the + // shape that carried them, which is strictly stronger, because there is no + // longer a mapping shape to author the key INTO. The base tombstone on + // `shared/FieldMapping` and the `integration/ConnectorFieldMapping` spelling + // are untouched and still reject the key with the #5552 prescription. + 'data/ExternalFieldMapping', + // #8075 — data/external-lookup.zod.ts, retired whole (ADR-0049). The root of + // the module's dependency shape: it embedded `ExternalDataSource` (the inline + // credential sink) and `ExternalFieldMapping`, and was itself referenced by no + // stack collection, metadata type or import outside packages/spec — the + // #5552 conversion's docblock had already recorded that no external-lookup + // document exists for the conversion walker to visit. Real-time external + // lookup returns via the enforce route of ADR-0049: the executor first, the + // vocabulary second. + 'data/ExternalLookup', // #4914 — the plugin manifest's `loading` block (ADR-0049 enforce-or-remove, // maintainer ruling 2026-08-04). `PluginLoadingConfig` was reachable from // authored metadata ONLY through `Manifest.loading`, and the ten members @@ -4716,11 +4818,48 @@ export const RETIRED_DEFS_BY_MAJOR: Readonly> 'kernel/PluginPreloadConfig', 'kernel/PluginSandboxing', 'shared/FieldMappingTransform', + // #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). Consumer + // group config (groupId / offset reset / auto-commit / poll size), embedded + // only by `system/MessageQueueConfig` (retired in the same change) and + // consumed by nothing. + 'system/ConsumerConfig', + // #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). DLQ config, + // embedded only by `system/MessageQueueConfig` (retired in the same change) + // and consumed by nothing. ⚠️ `kernel/DeadLetterQueueEntry` is a different + // declaration (the event bus's per-event DLQ record) and is untouched — + // name adjacency is not evidence, the `system/ServerRateLimitConfig` note + // applies verbatim. + 'system/DeadLetterQueue', + // #8075 — system/message-queue.zod.ts, retired whole (ADR-0049 + // enforce-or-remove; fork (b) of the #8075 census, accepted 2026-08-12). The + // top-level broker-administration config, carrying a required inline + // `sasl.password` credential when `sasl` was present. Zero consumers outside + // packages/spec repo-wide; no `message_queue` metadata type; the connector + // 'message_queue' ConnectorType enum value never referenced this schema. The + // consumed near-namesake `kernel/EventMessageQueueConfig` + // (`EventBusConfigSchema.messageQueue`) deliberately carries NO credential + // field and is untouched. Route 3: no tombstone, no D2 conversion — this + // table plus the D3 `external-lookup-message-queue-families-retired` are the + // declaration. + 'system/MessageQueueConfig', + // #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). The + // six-value broker provider enum, embedded only by + // `system/MessageQueueConfig` (retired in the same change). Not the same + // declaration as `kernel/EventMessageQueueConfig.provider` — that is its own + // INLINE `z.enum` (same six values, never a reference to this def) and is + // untouched. + 'system/MessageQueueProvider', // #5295 — system/http-server.zod.ts runtime vocabulary 'system/ServerCapabilities', 'system/ServerEvent', 'system/ServerEventType', 'system/ServerStatus', + // #8075 — system/message-queue.zod.ts, retired whole (ADR-0049). Topic + // partitions/replication/retention/compression config, embedded only by + // `system/MessageQueueConfig` (retired in the same change) and consumed by + // nothing — leaving it behind would strand an exported schema with no + // consumer (#3950). + 'system/TopicConfig', 'ui/DateFormat', 'ui/I18nObject', 'ui/LocaleConfig', diff --git a/packages/spec/src/shared/mapping.test.ts b/packages/spec/src/shared/mapping.test.ts index 2a59fb9320..69d2cb1d22 100644 --- a/packages/spec/src/shared/mapping.test.ts +++ b/packages/spec/src/shared/mapping.test.ts @@ -1,7 +1,6 @@ import { describe, it, expect } from 'vitest'; import { FieldMappingSchema } from './mapping.zod'; import { ConnectorFieldMappingSchema } from '../integration/connector.zod'; -import { ExternalFieldMappingSchema } from '../data/external-lookup.zod'; describe('FieldMappingSchema', () => { it('should accept minimal valid mapping', () => { @@ -95,14 +94,16 @@ describe('[#5552] FieldMapping.transform is retired, and says so', () => { expect(result.error!.issues[0]!.message).toMatch(/os migrate meta --from 16/s); }); - it('both extenders inherit the tombstone — one retirement, three authorable spellings', () => { - // `ConnectorFieldMappingSchema` and `ExternalFieldMappingSchema` are - // `.extend()`s of the base, so the retired property is copied into their - // shapes. This is why `RETIRED_KEYS_BY_MAJOR` registers three keys for one - // tombstone; if `.extend()` ever stopped copying it, this goes red. + it('the surviving extender inherits the tombstone — one retirement, two authorable spellings', () => { + // `ConnectorFieldMappingSchema` is an `.extend()` of the base, so the + // retired property is copied into its shape. This is why + // `RETIRED_KEYS_BY_MAJOR` registers a key per walked shape — nothing + // radiates from the base; if `.extend()` ever stopped copying it, this + // goes red. (`ExternalFieldMappingSchema` was the third spelling until the + // whole external-lookup family left in #8075; its retired-keys entry was + // subsumed by the def retirement, the WidgetManifest.performance way.) for (const [name, schema] of [ ['ConnectorFieldMapping', ConnectorFieldMappingSchema], - ['ExternalFieldMapping', ExternalFieldMappingSchema], ] as const) { const result = schema.safeParse(RETIRED); expect(result.success, `${name} must reject the retired key`).toBe(false); diff --git a/packages/spec/src/shared/mapping.zod.ts b/packages/spec/src/shared/mapping.zod.ts index 073225c3ac..adb6df247f 100644 --- a/packages/spec/src/shared/mapping.zod.ts +++ b/packages/spec/src/shared/mapping.zod.ts @@ -6,14 +6,16 @@ import { retiredKey } from './retired-key'; /** * Base Field Mapping Protocol * - * Shared by: Connector, External Lookup + * Shared by: Connector * * This module provides the canonical field mapping schema used across * ObjectStack for data synchronization. * * **Use Cases:** * - Integration connectors (integration/connector.zod.ts) - * - External lookups (data/external-lookup.zod.ts) + * - (`data/external-lookup.zod.ts`'s `ExternalFieldMappingSchema` extended + * this base too, until the whole external-lookup family was retired in + * #8075 — ADR-0049, zero consumers.) * * @example Basic field mapping * ```typescript diff --git a/packages/spec/src/system/index.ts b/packages/spec/src/system/index.ts index 7f6cc704e9..f6d9f1ff95 100644 --- a/packages/spec/src/system/index.ts +++ b/packages/spec/src/system/index.ts @@ -13,7 +13,22 @@ // Infrastructure Services export * from './cache.zod'; export * from './disaster-recovery.zod'; -export * from './message-queue.zod'; +// message-queue.zod (MessageQueueProviderSchema / TopicConfigSchema / +// ConsumerConfigSchema / DeadLetterQueueSchema / MessageQueueConfigSchema +// + every type alias) was REMOVED per ADR-0049 enforce-or-remove (#8075). The +// module declared broker-administration config for six providers — partitions, +// consumer groups, DLQs — including a required inline `sasl.password` +// credential, and no runtime ever parsed or read any of it: zero consumers +// outside this package repo-wide, no `message_queue` metadata type, and the +// connector 'message_queue' ConnectorType value never referenced these shapes. +// The consumed near-namesake is `kernel/events/integrations.zod.ts`'s +// `EventMessageQueueConfigSchema` (`EventBusConfigSchema.messageQueue`), which +// deliberately carries NO credential field — broker connection/credentials are +// deployment configuration for the runtime, not authorable metadata. Queue +// administration as authorable protocol metadata returns via the enforce route +// of ADR-0049 through a new ADR — the broker admin service first, the +// vocabulary second. See the D3 record +// `external-lookup-message-queue-families-retired`. export * from './object-storage.zod'; export * from './search-engine.zod'; export * from './http-server.zod'; diff --git a/packages/spec/src/system/message-queue-retirement.test.ts b/packages/spec/src/system/message-queue-retirement.test.ts new file mode 100644 index 0000000000..b7e1baa022 --- /dev/null +++ b/packages/spec/src/system/message-queue-retirement.test.ts @@ -0,0 +1,169 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; + +import { + EXPORT_ENTRY_POINTS, + exportNamesOf, + holdersOf, +} from '../../scripts/lib/export-origins-testkit'; + +// ─── [#8075] the message-queue config family is RETIRED ───────────────────── +// +// ADR-0049 enforce-or-remove, fork (b) of the #8075 census (accepted +// 2026-08-12): `system/message-queue.zod.ts` is deleted whole — 5 emitted defs +// (`system/MessageQueueConfig`, `system/MessageQueueProvider`, +// `system/TopicConfig`, `system/ConsumerConfig`, `system/DeadLetterQueue`), +// 14 exported names, reference docs with them. +// +// The measurement that decided it (issue #8075, report comment; spot-verified +// on the merged tree before this removal, control passing in the SAME run): +// +// 1. STATIC — zero consumers outside `packages/spec` repo-wide (definition + +// own tests + generated artifacts only), while the corpus-reach control +// (`DatasourceSchema` under identical exclusions) returns hits. The +// kernel "hits" for the name are the SUBSTRING inside +// `EventMessageQueueConfigSchema` — a different declaration. +// 2. DOORS — `DEFAULT_METADATA_TYPE_REGISTRY` has no `message_queue` type; +// `integration/connector.zod.ts`'s `'message_queue'` is only a +// `ConnectorType` enum VALUE (#7990's already-measured surface) and never +// referenced these shapes. +// 3. The security face: `MessageQueueConfigSchema.sasl` required an inline +// `password` whenever present — a broker credential in authorable +// metadata — while the CONSUMED near-namesake +// (`kernel/EventMessageQueueConfig`, `EventBusConfig.messageQueue`) +// deliberately carries no credential field. The consumed MQ shape has no +// credential key; the credential-bearing MQ shape had no consumer. +// +// ## Why route 3, and why there is nothing to tombstone +// +// With no carrier key there is no shape on which a `retiredKey()` tombstone +// could sit, and no author document for an ADR-0087 D2 conversion to rewrite — +// a prescription nobody can receive is noise. The declared record is the D3 +// `SemanticMigration` `external-lookup-message-queue-families-retired` plus +// the `RETIRED_DEFS_BY_MAJOR[17]` entries the manifest-deletion gate reads. +// +// Form follows #4988 / #5055: resolved symbol identity over every public entry +// via the build-time `export-origins/` artifact, plus the file-deletion probe +// in the #4988 direction (whole-file retirement, no surviving occupant). +describe('[#8075] system/ message-queue config family retirement', () => { + /** The 14 names the five retired defs exported (5 schema consts + 9 types). */ + const RETIRED_NAMES = [ + 'MessageQueueProviderSchema', 'MessageQueueProvider', + 'TopicConfigSchema', 'TopicConfig', 'TopicConfigParsed', + 'ConsumerConfigSchema', 'ConsumerConfig', 'ConsumerConfigParsed', + 'DeadLetterQueueSchema', 'DeadLetterQueue', 'DeadLetterQueueParsed', + 'MessageQueueConfigSchema', 'MessageQueueConfig', 'MessageQueueConfigParsed', + ] as const; + + /** + * The near-namesakes a "finish everything message-queue" sweep would + * plausibly take, each a DIFFERENT declaration with a live consumer: + * the event bus's MQ integration (no credential field, inline provider + * enum) and its per-event DLQ record. + */ + const MUST_SURVIVE_KERNEL = [ + 'EventMessageQueueConfigSchema', 'EventMessageQueueConfig', 'EventMessageQueueConfigParsed', + 'EventBusConfigSchema', + 'DeadLetterQueueEntrySchema', 'DeadLetterQueueEntry', 'DeadLetterQueueEntryParsed', + ] as const; + + /** System-entry neighbours that stay. */ + const MUST_SURVIVE_SYSTEM = [ + 'CacheConfigSchema', + 'CacheTierSchema', + ] as const; + + it('every retired name has ZERO holders on any public entry; the survivors still stand', () => { + // Anti-vacuity: the baseline must cover the real surface. + for (const needed of ['.', './system', './kernel', './data']) { + expect(EXPORT_ENTRY_POINTS, `exports map must include ${needed}`).toContain(needed); + } + expect(exportNamesOf('./system').length, './system must export a non-trivial surface').toBeGreaterThan(100); + + // ── ABSENCE (every entry — exact names, so the near-namesakes cannot + // satisfy these by substring) ──────────────────────────────────────── + for (const name of RETIRED_NAMES) { + expect(holdersOf(name), `${name} must have zero holders after #8075`).toEqual([]); + } + + // ── SURVIVAL ────────────────────────────────────────────────────────── + const kernelNames = exportNamesOf('./kernel'); + for (const name of MUST_SURVIVE_KERNEL) { + expect(kernelNames, `${name} must SURVIVE this retirement`).toContain(name); + } + const systemNames = exportNamesOf('./system'); + for (const name of MUST_SURVIVE_SYSTEM) { + expect(systemNames, `${name} must SURVIVE this retirement`).toContain(name); + } + }); + + it('the module is gone from disk, and nothing imports it any more', async () => { + const fs = await import('node:fs'); + const path = await import('node:path'); + const { fileURLToPath } = await import('node:url'); + const srcRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); + + for (const f of ['message-queue.zod.ts', 'message-queue.test.ts']) { + expect(fs.existsSync(path.join(srcRoot, 'system', f)), `system/${f} must be deleted`).toBe(false); + } + // Anti-vacuity: a kept sibling proves the probe looks in the right place. + expect(fs.existsSync(path.join(srcRoot, 'system', 'cache.zod.ts'))).toBe(true); + + const importers: string[] = []; + const walk = (dir: string) => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) walk(full); + else if (entry.name.endsWith('.ts')) { + const src = fs.readFileSync(full, 'utf-8'); + if (/(?:import|export)[^;]*['"][^'"]*\/message-queue\.zod(?:\.js)?['"]/.test(src)) { + importers.push(path.relative(srcRoot, full)); + } + } + } + }; + walk(srcRoot); + expect(importers, 'a resurrected import means the retirement is being undone — re-read #8075').toEqual([]); + }); + + it('runtime namespace agrees with the compiler view', async () => { + const system = await import('./index'); + for (const name of RETIRED_NAMES) { + expect(name in system, `system must not export ${name}`).toBe(false); + } + for (const name of MUST_SURVIVE_SYSTEM) { + expect(name in system, `${name} must SURVIVE at runtime`).toBe(true); + } + const kernel = await import('../kernel/index'); + for (const name of MUST_SURVIVE_KERNEL.filter((n) => n.endsWith('Schema'))) { + expect(name in kernel, `${name} must SURVIVE at runtime`).toBe(true); + } + }); + + it('the consumed MQ shape still carries NO credential face', async () => { + // The retirement's argument in one assertion: the LIVE message-queue + // surface (`EventBusConfig.messageQueue`) parses real configs and has no + // sasl / password / username slot. If someone re-adds a credential face + // here, this pin asks for the #7990 / #8075 analysis to be re-run, not + // for a quiet green. + const { EventMessageQueueConfigSchema } = await import('../kernel/events/integrations.zod'); + const parsed = EventMessageQueueConfigSchema.parse({ + provider: 'kafka', + topic: 'objectstack_events', + }); + expect(parsed.provider).toBe('kafka'); + // Defaults applied — the shape is genuinely parsed, not passed through. + expect(parsed.format).toBe('json'); + + // A config smuggling SASL credentials is not accepted-with-secrets: the + // schema is not `.strict()`, so zod STRIPS the unknown key — nothing + // credential-shaped survives into the parsed value. + const smuggled = EventMessageQueueConfigSchema.parse({ + provider: 'kafka', + topic: 'objectstack_events', + sasl: { mechanism: 'plain', username: 'u', password: 'p' }, + }); + expect(smuggled).not.toHaveProperty('sasl'); + }); +}); diff --git a/packages/spec/src/system/message-queue.test.ts b/packages/spec/src/system/message-queue.test.ts deleted file mode 100644 index 1d856f5c87..0000000000 --- a/packages/spec/src/system/message-queue.test.ts +++ /dev/null @@ -1,202 +0,0 @@ -import { describe, it, expect } from 'vitest'; -import { - MessageQueueProviderSchema, - TopicConfigSchema, - ConsumerConfigSchema, - DeadLetterQueueSchema, - MessageQueueConfigSchema, -} from './message-queue.zod'; - -describe('MessageQueueProviderSchema', () => { - it('should accept valid providers', () => { - const providers = ['kafka', 'rabbitmq', 'aws-sqs', 'redis-pubsub', 'google-pubsub', 'azure-service-bus']; - - providers.forEach((provider) => { - expect(() => MessageQueueProviderSchema.parse(provider)).not.toThrow(); - }); - }); - - it('should reject invalid providers', () => { - expect(() => MessageQueueProviderSchema.parse('invalid')).toThrow(); - expect(() => MessageQueueProviderSchema.parse('nats')).toThrow(); - }); -}); - -describe('TopicConfigSchema', () => { - it('should accept valid topic with defaults', () => { - const topic = TopicConfigSchema.parse({ - name: 'user_events', - }); - - expect(topic.name).toBe('user_events'); - expect(topic.partitions).toBe(1); - expect(topic.replicationFactor).toBe(1); - expect(topic.compressionType).toBe('none'); - }); - - it('should accept full topic configuration', () => { - const topic = TopicConfigSchema.parse({ - name: 'order_events', - partitions: 12, - replicationFactor: 3, - retentionMs: 604800000, - compressionType: 'snappy', - }); - - expect(topic.partitions).toBe(12); - expect(topic.replicationFactor).toBe(3); - expect(topic.retentionMs).toBe(604800000); - expect(topic.compressionType).toBe('snappy'); - }); - - it('should accept all compression types', () => { - const types = ['none', 'gzip', 'snappy', 'lz4']; - - types.forEach((type) => { - expect(() => TopicConfigSchema.parse({ name: 'test', compressionType: type })).not.toThrow(); - }); - }); - - it('should reject missing required name', () => { - expect(() => TopicConfigSchema.parse({})).toThrow(); - }); - - it('should reject invalid compression type', () => { - expect(() => TopicConfigSchema.parse({ name: 'test', compressionType: 'zstd' })).toThrow(); - }); -}); - -describe('ConsumerConfigSchema', () => { - it('should accept valid consumer with defaults', () => { - const consumer = ConsumerConfigSchema.parse({ - groupId: 'order_processor', - }); - - expect(consumer.groupId).toBe('order_processor'); - expect(consumer.autoOffsetReset).toBe('latest'); - expect(consumer.enableAutoCommit).toBe(true); - expect(consumer.maxPollRecords).toBe(500); - }); - - it('should accept full consumer configuration', () => { - const consumer = ConsumerConfigSchema.parse({ - groupId: 'analytics_consumer', - autoOffsetReset: 'earliest', - enableAutoCommit: false, - maxPollRecords: 1000, - }); - - expect(consumer.autoOffsetReset).toBe('earliest'); - expect(consumer.enableAutoCommit).toBe(false); - expect(consumer.maxPollRecords).toBe(1000); - }); - - it('should accept all autoOffsetReset values', () => { - const values = ['earliest', 'latest']; - - values.forEach((value) => { - expect(() => ConsumerConfigSchema.parse({ groupId: 'test', autoOffsetReset: value })).not.toThrow(); - }); - }); - - it('should reject missing required groupId', () => { - expect(() => ConsumerConfigSchema.parse({})).toThrow(); - }); -}); - -describe('DeadLetterQueueSchema', () => { - it('should accept valid DLQ with defaults', () => { - const dlq = DeadLetterQueueSchema.parse({ - queueName: 'failed_messages', - }); - - expect(dlq.queueName).toBe('failed_messages'); - expect(dlq.enabled).toBe(false); - expect(dlq.maxRetries).toBe(3); - }); - - it('should accept full DLQ configuration', () => { - const dlq = DeadLetterQueueSchema.parse({ - enabled: true, - maxRetries: 5, - queueName: 'order_dlq', - }); - - expect(dlq.enabled).toBe(true); - expect(dlq.maxRetries).toBe(5); - expect(dlq.queueName).toBe('order_dlq'); - }); - - it('should reject missing required queueName', () => { - expect(() => DeadLetterQueueSchema.parse({})).toThrow(); - expect(() => DeadLetterQueueSchema.parse({ enabled: true })).toThrow(); - }); -}); - -describe('MessageQueueConfigSchema', () => { - it('should accept minimal config with defaults', () => { - const config = MessageQueueConfigSchema.parse({ - provider: 'kafka', - topics: [{ name: 'events' }], - }); - - expect(config.provider).toBe('kafka'); - expect(config.topics).toHaveLength(1); - expect(config.ssl).toBe(false); - }); - - it('should accept full configuration', () => { - const config = MessageQueueConfigSchema.parse({ - provider: 'kafka', - topics: [ - { name: 'user_events', partitions: 6, replicationFactor: 3 }, - { name: 'order_events', partitions: 12, compressionType: 'gzip' }, - ], - consumers: [ - { groupId: 'user_processor' }, - { groupId: 'order_processor', autoOffsetReset: 'earliest' }, - ], - deadLetterQueue: { - enabled: true, - maxRetries: 5, - queueName: 'dlq', - }, - ssl: true, - sasl: { - mechanism: 'scram-sha-256', - username: 'admin', - password: 'secret', - }, - }); - - expect(config.topics).toHaveLength(2); - expect(config.consumers).toHaveLength(2); - expect(config.deadLetterQueue?.enabled).toBe(true); - expect(config.ssl).toBe(true); - expect(config.sasl?.mechanism).toBe('scram-sha-256'); - }); - - it('should accept all SASL mechanisms', () => { - const mechanisms = ['plain', 'scram-sha-256', 'scram-sha-512']; - - mechanisms.forEach((mechanism) => { - expect(() => MessageQueueConfigSchema.parse({ - provider: 'kafka', - topics: [{ name: 'test' }], - sasl: { mechanism, username: 'user', password: 'pass' }, - })).not.toThrow(); - }); - }); - - it('should reject missing required fields', () => { - expect(() => MessageQueueConfigSchema.parse({})).toThrow(); - expect(() => MessageQueueConfigSchema.parse({ provider: 'kafka' })).toThrow(); - }); - - it('should reject invalid provider', () => { - expect(() => MessageQueueConfigSchema.parse({ - provider: 'invalid', - topics: [{ name: 'test' }], - })).toThrow(); - }); -}); diff --git a/packages/spec/src/system/message-queue.zod.ts b/packages/spec/src/system/message-queue.zod.ts deleted file mode 100644 index 22043ac6cc..0000000000 --- a/packages/spec/src/system/message-queue.zod.ts +++ /dev/null @@ -1,69 +0,0 @@ -// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. - -import { z } from 'zod'; - -/** - * Message queue protocol for async communication - * Supports Kafka, RabbitMQ, AWS SQS, Redis Pub/Sub - */ -import { lazySchema } from '../shared/lazy-schema'; -export const MessageQueueProviderSchema = lazySchema(() => z.enum([ - 'kafka', - 'rabbitmq', - 'aws-sqs', - 'redis-pubsub', - 'google-pubsub', - 'azure-service-bus', -]).describe('Supported message queue backend provider')); - -export type MessageQueueProvider = z.input; - -export const TopicConfigSchema = lazySchema(() => z.object({ - name: z.string().describe('Topic name identifier'), - partitions: z.number().default(1).describe('Number of partitions for parallel consumption'), - replicationFactor: z.number().default(1).describe('Number of replicas for fault tolerance'), - retentionMs: z.number().optional().describe('Message retention period in milliseconds'), - compressionType: z.enum(['none', 'gzip', 'snappy', 'lz4']).default('none').describe('Message compression algorithm'), -}).describe('Configuration for a message queue topic')); - -export type TopicConfig = z.input; -/** Post-parse shape of {@link TopicConfig} — defaults applied, transforms run (ADR-0122). */ -export type TopicConfigParsed = z.infer; - -export const ConsumerConfigSchema = lazySchema(() => z.object({ - groupId: z.string().describe('Consumer group identifier'), - autoOffsetReset: z.enum(['earliest', 'latest']).default('latest').describe('Where to start reading when no offset exists'), - enableAutoCommit: z.boolean().default(true).describe('Automatically commit consumed offsets'), - maxPollRecords: z.number().default(500).describe('Maximum records returned per poll'), -}).describe('Consumer group configuration for topic consumption')); - -export type ConsumerConfig = z.input; -/** Post-parse shape of {@link ConsumerConfig} — defaults applied, transforms run (ADR-0122). */ -export type ConsumerConfigParsed = z.infer; - -export const DeadLetterQueueSchema = lazySchema(() => z.object({ - enabled: z.boolean().default(false).describe('Enable dead letter queue for failed messages'), - maxRetries: z.number().default(3).describe('Maximum delivery attempts before sending to DLQ'), - queueName: z.string().describe('Name of the dead letter queue'), -}).describe('Dead letter queue configuration for unprocessable messages')); - -export type DeadLetterQueue = z.input; -/** Post-parse shape of {@link DeadLetterQueue} — defaults applied, transforms run (ADR-0122). */ -export type DeadLetterQueueParsed = z.infer; - -export const MessageQueueConfigSchema = lazySchema(() => z.object({ - provider: MessageQueueProviderSchema.describe('Message queue backend provider'), - topics: z.array(TopicConfigSchema).describe('List of topic configurations'), - consumers: z.array(ConsumerConfigSchema).optional().describe('Consumer group configurations'), - deadLetterQueue: DeadLetterQueueSchema.optional().describe('Dead letter queue for failed messages'), - ssl: z.boolean().default(false).describe('Enable SSL/TLS for broker connections'), - sasl: z.object({ - mechanism: z.enum(['plain', 'scram-sha-256', 'scram-sha-512']).describe('SASL authentication mechanism'), - username: z.string().describe('SASL username'), - password: z.string().describe('SASL password'), - }).optional().describe('SASL authentication configuration'), -}).describe('Top-level message queue configuration')); - -export type MessageQueueConfig = z.input; -/** Post-parse shape of {@link MessageQueueConfig} — defaults applied, transforms run (ADR-0122). */ -export type MessageQueueConfigParsed = z.infer; diff --git a/packages/spec/src/type-alias-convention.pin.test.ts b/packages/spec/src/type-alias-convention.pin.test.ts index afe3ee8595..15cc3b8306 100644 --- a/packages/spec/src/type-alias-convention.pin.test.ts +++ b/packages/spec/src/type-alias-convention.pin.test.ts @@ -139,7 +139,6 @@ import type * as M61 from './data/driver/common.zod.js'; import type * as M62 from './data/driver/memory.zod.js'; import type * as M63 from './data/driver/sqlite.zod.js'; import type * as M181 from './data/driver/turso.zod.js'; -import type * as M64 from './data/external-lookup.zod.js'; import type * as M65 from './data/feed.zod.js'; import type * as M66 from './data/field.zod.js'; import type * as M67 from './data/filter.zod.js'; @@ -226,7 +225,6 @@ import type * as M133 from './system/incident-response.zod.js'; import type * as M134 from './system/job.zod.js'; import type * as M135 from './system/license.zod.js'; import type * as M136 from './system/logging.zod.js'; -import type * as M137 from './system/message-queue.zod.js'; import type * as M138 from './system/metadata-persistence.zod.js'; import type * as M139 from './system/metrics.zod.js'; import type * as M140 from './system/migration.zod.js'; @@ -743,8 +741,7 @@ export type Iso334 = Assert, z.infer< typeof M181.TursoTransportModeSchema > >>; -// data/external-lookup.zod.ts -export type Iso335 = Assert, z.infer< typeof M64.ExternalDataSourceSchema > >>; +// data/external-lookup.zod.ts — retired whole (#8075, ADR-0049); its pin left with it. // data/feed.zod.ts export type Iso336 = Assert, z.infer< typeof M65.FeedItemType > >>; @@ -1153,8 +1150,7 @@ export type Iso583 = Assert, z.inf export type Iso584 = Assert, z.infer< typeof M136.ExternalServiceDestinationConfigSchema > >>; export type Iso585 = Assert, z.infer< typeof M136.StructuredLogEntrySchema > >>; -// system/message-queue.zod.ts -export type Iso586 = Assert, z.infer< typeof M137.MessageQueueProviderSchema > >>; +// system/message-queue.zod.ts — retired whole (#8075, ADR-0049); its pin left with it. // system/metadata-persistence.zod.ts export type Iso587 = Assert, z.infer< typeof M138.MetadataScopeSchema > >>; From bb9c4195a79e7db79fd80337d266d29c9bc43d40 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 12 Aug 2026 20:39:03 +0000 Subject: [PATCH 2/3] =?UTF-8?q?test(spec):=20close=20the=20retirement's=20?= =?UTF-8?q?trailing=20surfaces=20=E2=80=94=20pin=20count=20826=E2=86=92824?= =?UTF-8?q?,=20PROTOCOL=5FMAP=20rows,=20strictness-ledger=20rows=20(#8075)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0123k4cam2jEAkPmbJeoaY3r --- ...07-unknown-key-strictness-ledger.counts.md | 24 +++++++++---------- .../2026-07-unknown-key-strictness-ledger.md | 8 +++---- packages/spec/PROTOCOL_MAP.md | 2 -- .../src/type-alias-convention.pin.test.ts | 14 ++++++++--- 4 files changed, 26 insertions(+), 22 deletions(-) diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md index 67f97f89b9..96230e37e4 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md @@ -21,16 +21,16 @@ regenerate. | Measure | Value | |---|---| | Triaged directories | 5 | -| Object sites in them | 438 | -| Still-open (strip) sites | 144 | -| Files carrying at least one | 25 | +| Object sites in them | 426 | +| Still-open (strip) sites | 132 | +| Files carrying at least one | 24 | Remaining strip sites by class: | Bucket | Sites | |---|---| | authorable — the ruling's forced scope | 4 | -| unresolved — needs a per-schema verdict | 34 | +| unresolved — needs a per-schema verdict | 22 | | wire / open — out of forced scope | 104 | | no door — no carrier, ADR-0049 territory | 1 | | no gate — carrier live, no parse | 0 | @@ -45,11 +45,11 @@ The `strict` column is the one the campaign schedules against; it counts both th | Dir | Sites | strict | passthrough | catchall | strip | |---|---|---|---|---|---| | `ui/` | 161 | 150 | 5 | 0 | 6 | -| `data/` | 165 | 62 | 1 | 0 | 102 | +| `data/` | 153 | 62 | 1 | 0 | 90 | | `automation/` | 65 | 42 | 0 | 0 | 23 | | `security/` | 20 | 7 | 0 | 0 | 13 | | `studio/` | 27 | 27 | 0 | 0 | 0 | -| **total** | **438** | **288** | **6** | **0** | **144** | +| **total** | **426** | **288** | **6** | **0** | **132** | ## File-level triage — site counts @@ -97,7 +97,6 @@ classify and is not listed (it becomes reportable the day it grows its first sit | `driver/sqlite.zod.ts` | 2 | | `driver/turso.zod.ts` | 2 | | `external-catalog.zod.ts` | 4 | -| `external-lookup.zod.ts` | 12 | | `field-value.zod.ts` | 3 | | `field.zod.ts` | 10 | | `filter.zod.ts` | 11 | @@ -109,7 +108,7 @@ classify and is not listed (it becomes reportable the day it grows its first sit | `seed-loader.zod.ts` | 12 | | `seed.zod.ts` | 1 | | `validation.zod.ts` | 6 | -| **total** | **165** | +| **total** | **153** | ### `automation/` — sites @@ -178,7 +177,7 @@ over it is here. ### `data/` — open -**102 strip of 165**, in 15 file(s). +**90 strip of 153**, in 14 file(s). | File | Strip | Sites | |---|---|---| @@ -189,7 +188,6 @@ over it is here. | `driver-sql.zod.ts` | 2 | 2 | | `driver.zod.ts` | 9 | 9 | | `external-catalog.zod.ts` | 4 | 4 | -| `external-lookup.zod.ts` | 12 | 12 | | `field-value.zod.ts` | 2 | 3 | | `field.zod.ts` | 2 | 10 | | `filter.zod.ts` | 10 | 11 | @@ -197,12 +195,12 @@ over it is here. | `object.zod.ts` | 1 | 20 | | `query.zod.ts` | 4 | 5 | | `seed-loader.zod.ts` | 12 | 12 | -| **total** | **102** | **165** | +| **total** | **90** | **153** | | Bucket | Sites | |---|---| | authorable — the ruling's forced scope | 3 | -| unresolved — needs a per-schema verdict | 34 | +| unresolved — needs a per-schema verdict | 22 | | wire / open — out of forced scope | 65 | | no door — no carrier, ADR-0049 territory | 0 | | no gate — carrier live, no parse | 0 | @@ -269,4 +267,4 @@ directory rather than per file. | `kernel/` | 296 | | `qa/` | 6 | | `shared/` | 20 | -| `system/` | 365 | +| `system/` | 360 | diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.md b/docs/audits/2026-07-unknown-key-strictness-ledger.md index c2563b4af7..b8960e557b 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.md @@ -689,7 +689,6 @@ column does not move and the `strip` column falls by the count of what left. |---|---|---| | `object.zod.ts` | authorable | top-level already guarded (#1535); inner blocks strict as of 批 20 — except `IndexSchema`, held on a measured console drift (see the `data/` remaining-strip row) | | `data-engine.zod.ts` | wire (p) | engine contract shapes (was 14 — `DataEngineBatchRequestSchema` retired with `IDataEngine.batch?`, #4618) | -| `external-lookup.zod.ts` | mixed (p) | authored config + wire results | | `seed-loader.zod.ts` | mixed (p) | seed file shapes are authored; loader state is runtime | | `field.zod.ts` | authorable | partially strict | | `filter.zod.ts` | open | query dialect — user data flows through the predicate values; validated semantically elsewhere | @@ -1174,7 +1173,6 @@ triage row record which one was taken. |---|---|---| | `object.zod.ts` | authorable | **13 of 14 closed at #4001 批 20; the 14th is HELD, and that hold is the batch's finding.** The registered type's top level was already closed (#1535/#4519/#4522) — these were the inner blocks under it, and the asymmetry is why the file mattered: an author who has SEEN the root reject a typo reads a clean parse of `lifecycle: { maxAge: '30d' }` as acceptance. Closed: `ObjectAccessConfig`, `Lifecycle` + all four sub-blocks (`retention`/`ttl`/`storage`/`archive`), `ObjectFieldGroup`, `ObjectExternalBinding`, `userActions`, `systemFields`, `activityMilestones`, `publicSharing`, `ObjectExtension`. Reachability was measured, not assumed: a BFS from all 24 metadata-type roots plus `ObjectStackSchema` (4810 nodes, 25 roots) resolves every one `direct` **by identity** — none rests on the `derived-clone` bridge #5056 found can mark a dead shape reachable — with `ObjectSchema`/`PageSchema` as positive controls and 批 14's `EmbedConfigSchema` UNREACHABLE (overlap 0.00) **in the same run**. The parse door was probed separately and at each path: `ObjectSchema.safeParse` REJECTS at the top level today and SILENTLY STRIPPED at all thirteen nested paths, so each closure converts a measured silent strip, not a hypothesis. Doors: `saveMetaItem`'s 422 (`metadata-protocol/protocol.ts` — `getMetadataTypeSchema('object')`), `ObjectSchema.create()`, `defineObjectExtension()`, and `registry.validate()` (diagnostic-only by #3903 design). ⚠️ **`IndexSchema` is deliberately NOT closed — the #5114 class, caught before it shipped rather than after.** objectui's console ships its own hand-copied JSON-Schema for this shape (`metadata-admin/EmbeddedItemEditor.tsx` → `FALLBACK_SCHEMAS.index`), because `index` is an embedded-only sub-type the framework publishes no schema for; that copy has drifted and offers **`where`** for the partial-index predicate where this schema declares **`partial`** (and `brin` in an enum that has no `brin`). The editor splices its form output into `object.indexes[]` and PUTs the WHOLE object, and `saveMetaItem` keeps the body verbatim while validating it — so closing this one shape would 422 a control the console itself renders. Unlike #5073's `allowAddTab` the capability is not merely un-gated but already DEAD in both directions: `driver-sql`'s `syncDeclaredIndexes` consumes `name`/`fields`/`unique` only, so neither `where` NOR `partial` reaches any DDL. That is why the hold is not just "fix the producer first": pointing an author at `partial` today would be a guidance entry claiming more than the platform delivers (finding 18), so the close is gated on BOTH the objectui rename (**#5247**) and an ADR-0049 answer for `type`/`partial` (**#5248**). Recorded in three places (the `IndexSchema` JSDoc + `object-strictness-batch20.test.ts` §4 + this row). One caveat shipped knowingly: `systemFields` is a `false | {…}` union, so its rejection is an `invalid_union` whose own message is the bare *"Invalid input"* — the #5014 flattening. 批 18's `discriminatedUnion` fix is unavailable (one arm is a literal, so there is no discriminant), so the behaviour is pinned honestly rather than papered over; every other site in the file is a plain object and surfaces its prescription directly. Curation is anchored to named siblings and each claim is asserted: the dominant failure here is FLATTENING (`maxAge`/`expireAfter`/`shards` written one level too high, where §3.5's own refine then rejects the object for the WRONG key), so `lifecycle` carries wrong-layer pointers DOWN into its four sub-blocks; `userActions` points at `ui/view.zod.ts`'s identically-named block, whose vocabulary is completely disjoint; `systemFields.owner` points at `ownership`, a key the block's own field doc names but the shape never declared; `external.allowWrites` names the ADR-0015 double opt-in and mirrors `datasource.zod.ts`'s own `writable → allowWrites` alias in the opposite direction; `fieldGroups[].fields` states the direction of the membership edge (declared on the FIELD), and its three DEPRECATED collapse aliases stay ACCEPTED — closing a shape must not turn a documented deprecation into a rejection. Highest author volume in the repo | | `data-engine.zod.ts` | wire | **out of scope** — engine request/response contracts | -| `external-lookup.zod.ts` | mixed (p) | `ExternalDataSource` + `.authentication` and the `ExternalLookup` tree are authored config; needs the per-schema read the ledger never did | | `seed-loader.zod.ts` | mixed (p) | Split is real: `SeedLoaderConfig` / `SeedIdentity` (+`.user`/`.org`) / `ReferenceResolution` are authored; `SeedLoadResult` / `SeedLoaderResult` (+`.summary`) / `ReferenceResolutionError` / `ObjectDependencyNode` / `ObjectDependencyGraph` / `SeedLoaderRequest` are loader runtime | | `filter.zod.ts` | open | **out of scope** — query dialect; user data flows through, validated semantically elsewhere | | `driver-nosql.zod.ts` | wire | **out of scope** | @@ -1191,9 +1189,11 @@ triage row record which one was taken. **Authorable strip in `data/`:** [the counts file](./2026-07-unknown-key-strictness-ledger.counts.md#data--open) splits this directory three ways, and the middle bucket is the one to read: `object` and `field` are -**firm** authorable; `external-lookup`, `seed-loader`, `analytics` and `field-value` are +**firm** authorable; `seed-loader`, `analytics` and `field-value` are **unresolved** — they still carry `mixed (p)`, so the ledger is saying "nobody has done the -per-schema read" rather than "these are ready". The rest is wire/open and out of the +per-schema read" rather than "these are ready". (`external-lookup` carried `mixed (p)` too +until #8075 retired the whole file under ADR-0049 — its per-schema read arrived as a +zero-consumer verdict, and the strictness question died with the shapes.) The rest is wire/open and out of the ruling's forced scope; that count fell by one when #4721 closed `query.zod.ts`'s `SortNodeSchema`, the one row in this directory where the per-schema read moved a site OUT of `open` rather than confirming it, and by one more when **#4001 batch B** closed diff --git a/packages/spec/PROTOCOL_MAP.md b/packages/spec/PROTOCOL_MAP.md index ec08bb5a2d..c63d35bff4 100644 --- a/packages/spec/PROTOCOL_MAP.md +++ b/packages/spec/PROTOCOL_MAP.md @@ -34,7 +34,6 @@ This document serves as the **Grand Map** of the ObjectStack specification. It l | [`seed-loader.zod.ts`](src/data/seed-loader.zod.ts) | | **Seed Loader**. How seed rows are resolved and applied. | | [`analytics.zod.ts`](src/data/analytics.zod.ts) | | **Data Analytics**. Aggregation and multidimensional analysis types. | | [`document.zod.ts`](src/data/document.zod.ts) | | **Document**. Unstructured document storage protocol. | -| [`external-lookup.zod.ts`](src/data/external-lookup.zod.ts) | | **External Lookup**. Virtual lookups to external API data. | | [`filter.zod.ts`](src/data/filter.zod.ts) | | **Filter**. Low-level filter syntax definitions. | | [`hook.zod.ts`](src/data/hook.zod.ts) | | **Triggers/Hooks**. Database trigger definitions (before/after insert/update). | | [`mapping.zod.ts`](src/data/mapping.zod.ts) | | **Data Mapping**. Rules for transforming data between schemas. | @@ -131,7 +130,6 @@ This document serves as the **Grand Map** of the ObjectStack specification. It l | [`metrics.zod.ts`](src/system/metrics.zod.ts) | | **Observability**. Prometheus/OpenTelemetry metrics. | | [`tracing.zod.ts`](src/system/tracing.zod.ts) | | **Tracing**. Distributed tracing configuration. | | [`object-storage.zod.ts`](src/system/object-storage.zod.ts) | | **File Storage**. S3 bucket and upload configurations. | -| [`message-queue.zod.ts`](src/system/message-queue.zod.ts) | | **Message Queue**. Internal event bus settings. | | [`search-engine.zod.ts`](src/system/search-engine.zod.ts) | | **Search**. Elasticsearch/Meilisearch configuration. | | [`notification.zod.ts`](src/system/notification.zod.ts) | | **Notifications**. System-wide notification preferences. | | [`translation.zod.ts`](src/system/translation.zod.ts) | | **i18n**. Internationalization and localization. | diff --git a/packages/spec/src/type-alias-convention.pin.test.ts b/packages/spec/src/type-alias-convention.pin.test.ts index 15cc3b8306..a7560dc395 100644 --- a/packages/spec/src/type-alias-convention.pin.test.ts +++ b/packages/spec/src/type-alias-convention.pin.test.ts @@ -263,7 +263,7 @@ import type * as M167 from './ui/view.zod.js'; import type * as M170 from './ui/component.zod.js'; // --------------------------------------------------------------------------- -// 826 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. +// 824 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. // // That number is machine-checked, not hand-kept. The runtime companion at the // bottom of this file recomputes the pin count from the source and asserts that @@ -1621,7 +1621,7 @@ describe('ADR-0122 type-alias convention', () => { // this title and the section header above the pin list — are now asserted // against the recomputed count below, so neither can go stale without a red // test naming it. - it('still declares all 826 isomorphic pins', () => { + it('still declares all 824 isomorphic pins', () => { // The truth of each pin is proved by tsc, not here — an `Assert>` // that stops holding is a compile error with the alias named. What tsc // cannot notice is a pin that was DELETED: removing the assertion removes @@ -1808,9 +1808,17 @@ describe('ADR-0122 type-alias convention', () => { // widens input and output identically, so `Iso136` still holds and the // count moves by exactly the one new schema. Its id is `Iso837`, the next // free one — the ids are claims about pins, not positions. + // + // 826 -> 824 is #8075's ADR-0049 retirement of two whole modules: + // `data/external-lookup.zod.ts` (`Iso335`, `ExternalDataSourceSchema`) and + // `system/message-queue.zod.ts` (`Iso586`, `MessageQueueProviderSchema`). + // A pin leaves when its schema leaves — the schemas are deleted, so the + // pins are deleted with them, not re-pointed. The ids `Iso335`/`Iso586` + // are retired with their subjects and are NOT free for reuse: the ids are + // claims about pins, not positions. const self = readFileSync(fileURLToPath(import.meta.url), 'utf8'); const pins = self.match(/^export type Iso\d+ = Assert Date: Wed, 12 Aug 2026 22:06:53 +0000 Subject: [PATCH 3/3] regen: rebuild generated artifacts from the merged tree (os-regen landing lap, #8075) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merge a296d817e took main's side of every os-regen path; this regeneration recomputes them from the merged sources — the 8 retired defs and 39 authorable keys leave again (deliberate deletions re-applied under the manifest gate), main's audit-log semantic entry, page.zod/editability references and the build-docs integration fix (#8219) all survive. check:generated 13/13. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_0123k4cam2jEAkPmbJeoaY3r --- docs/protocol-upgrade-guide.md | 3 +++ packages/spec/spec-changes.json | 14 ++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index c0b2bc580b..a44a0b5e7f 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -328,6 +328,9 @@ The action LOCATION vocabulary loses `global_nav` in this step (#6888, ADR-0049, - **`apimethod-enum-shrink`** — `data.object.enable.apiMethods (the eight legacy non-primitive values)` → the six primitives only — `get` / `list` / `create` / `update` / `delete` / `bulk`: replace each legacy value with the primitives it derives from, de-duplicate, and delete the key entirely if the result names all six - Why not automatic: The authored `enable.apiMethods` enum is now exactly the six primitives. The eight legacy values — `upsert`, `aggregate`, `history`, `search`, `restore`, `purge`, `import`, `export` — are no longer authorable, because they are DERIVED effective operations resolved by the server's single derivation table, and an enum that lets an author name both a primitive and something derived from it has two spellings for one fact. The FROM → TO is a table rather than a rename: `upsert` → `create` + `update`; `import` → `create` + `update`; `export`, `aggregate` and `search` → `list`; `history` → `get`; and `restore` / `purge` map to NOTHING — they never derived, because `enable.trash` was retired in #2377, so the value is deleted outright. That last row is why this is a semantic entry and not a mechanical conversion, and the reason is a security one: the mapping WIDENS. An allowlist naming `history` was granting read of one record's audit trail; rewritten to `get` it grants ordinary record reads, and an allowlist naming `search` becomes a grant of full `list`. A transform that applied the table silently would broaden real API permissions without anyone reading the diff, so the rewrite is delegated to the author with the widening flagged. The reporter codemod exists for exactly that shape: `node scripts/codemod/apimethods-legacy-to-primitives.mjs` scans, reports the exact replacement per site, and FLAGS the allowlists the mapping would widen so the edit stays reviewable — it reports, it does not rewrite. Stored metadata keeps parsing (permanent tolerance, narrowing only), so nothing breaks at rest; what changes is what an author may newly write. Registered by the #6350 stock reconciliation; #3543 (P2 of #3391) predates the #6148 completeness gate. ADR-0087, #3543 (backfilled #6350). - Done when: No authored `enable.apiMethods` array names a legacy value; `objectstack validate` passes. Run the reporter codemod first and read its widening flags before applying anything — ⚠️ the migration is only correct if each widened grant was INTENDED. For every object where `history` became `get` or `search` became `list`, confirm the broader operation is one the API should genuinely expose; where it is not, the answer is not a different value in this enum but a permission set that withholds the operation. Where the six primitives are all present, prefer deleting the key: that is equivalent to default-open and it tracks future primitives, whereas a hand-listed six silently stops granting anything added later. `restore` / `purge` are deleted with no replacement — if trash-like behaviour was being relied on, that capability left in #2377 and this entry is not where it returns. +- **`audit-log-action-enum-retired`** — `sys_audit_log.action — the values 'export' and 'permission_change' left the select enum declared by plugin-audit (packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts). The same two values also left the shipped list-view filters on that object: 'permission_change' from the auth_events view and 'export' from the config_changes view` → nothing, for either value — both are removed rather than renamed, because neither named an event this platform records. For permission changes, read the ordinary `create` / `update` rows on the permission objects themselves: a grant or binding write is an ordinary record write and the generic audit writer already ledgers it, so a second semantically-duplicate row was never minted. For `export` there is no replacement and nothing is lost: no export feature ever wrote an audit row. A consumer filtering `sys_audit_log` on either value was reading an empty result set on every deployment, and still is — what changed is that the contract no longer promises otherwise + - Why not automatic: Maintainer ruling 2026-08-12 (#7675), the retirement half of a two-half verdict: the cheap writers get built (#8144 login/logout, #8145 config_change) and the enum values with no feature behind them are retired. 原则记录:空 widget + 永远查不到东西的过滤器是可见产品缺陷;审计面宁窄勿谎. The defect was false compliance on a COMPLIANCE surface, which is the sharpest form of ADR-0049 declared-≠-enforced: an auditor reading the action enum believed the platform captured permission changes and data exports, and the shipped list views and dashboard widgets showed them a filter and a tile for exactly those events. Both were permanently empty. Measured by enumerating every `sys_audit_log` writer in the repo — there are exactly two: plugin-audit`s generic hook writer, whose `actionFor` maps afterInsert/Update/Delete to create/update/delete and nothing else, and plugin-auth`s admin user-import. Neither has ever emitted `export` or `permission_change`. This is an enum-VALUE retirement, so the bookkeeping differs from a key retirement in the two ways `hook-body-crypto-hash-removed`, `dataset-measure-array-string-agg-removed` and `action-global-nav-location-removed` already record: nothing lands in RETIRED_KEYS_BY_MAJOR (no authorable KEY changed) and the four surface ratchets are expected to be byte-identical (no def changed). It differs from all three in being a SEMANTIC entry rather than a D2 conversion, and the reason is that there is no source to rewrite: `sys_audit_log` is a platform-owned, append-only object whose every field is `readonly: true`. Nobody authors an audit row and nobody authors this enum — the values appear only in rows the runtime writes and in queries consumers send. A conversion rewrites authored metadata or a stored `sys_metadata` row; this surface is neither, so the disposition is the one `BatchOptions.validateOnly` and the notification cursor already take in this major. ⚠️ Historical ROWS are deliberately untouched. A deployment that somehow holds a row with either value keeps it, and keeps reading it back: the enum is not enforced on this object at all (`validateRecord` skips `readonly` fields, and every field here is readonly), so nothing rejects stored history and no backfill is required or wanted. Deleting audit history to satisfy a schema narrowing would be the one genuinely destructive reading of this change. ADR-0049 / ADR-0087, #8147. + - Done when: No consumer filters `sys_audit_log` on `action = "export"` or `action = "permission_change"` expecting rows: both were empty everywhere before this change, so a query that returned data has not been identified and a query that returned nothing behaves identically. Concretely, check three places. (1) Saved queries, dashboards and reports over `sys_audit_log`: a filter naming either value should be deleted, not re-pointed — for permission auditing, filter the permission objects` own `create`/`update` rows by `object_name` instead. (2) Any code branching on the action string (a badge map, a label switch, an `if (row.action === ...)`): the arms for these two values are now unreachable and should go, and a `switch` with an exhaustiveness check over the enum type will now fail to compile if they stay — that compile error is the enforced channel for TypeScript consumers. (3) Custom objects or plugins inserting `sys_audit_log` rows with either value: this is the only case that needs a real decision, because the write will NOT be refused (readonly fields are not validated) — it will simply be a row whose action the object no longer declares. Pick a declared value or open an issue for the action you actually need. ⚠️ Do NOT migrate or delete existing rows: audit history is append-only and stays exactly as written. - **`auth-config-unadvertised-reserved-features`** — `api.authConfig.features.passkeys / api.authConfig.features.magicLink` → (removed — no replacement flag; the capabilities are not advertised) - Why not automatic: Both flags were served by `GET /api/v1/auth/config` from introduction and read by no client: no login UI anywhere renders a passkey or magic-link affordance off them, so the payload advertised two sign-in methods a user could never reach, and a deployer setting `plugins.passkeys` / `plugins.magicLink` flipped a switch with no observable effect (ADR-0049 enforce-or-remove; maintainer ruling 2026-08-11 on #7481 chose remove over keep-as-reserved). The two are not equally empty: nothing at all is wired behind `passkeys`, whereas `magicLink`'s better-auth endpoints are live and only their advertisement was withdrawn. This is a RESPONSE surface — nobody authors or persists an `AuthFeaturesConfig` — so there is no source for the chain to rewrite; the schema tombstones both keys via retiredKey() and consumers drop their read. The withdrawal is conditional: both return to the payload in the change that ships the login UI (objectui#4179). ADR-0049, #7481. - Done when: No client reads `features.passkeys` or `features.magicLink` off `/api/v1/auth/config`; a client that gated UI on either now treats the capability as absent rather than reading `undefined` as false by accident, and constructing an `AuthFeaturesConfig` with either key fails to parse with its own prescription instead of being silently stripped. Magic-link deployments keep working: `plugins.magicLink` still mounts `/api/v1/auth/magic-link/send` and `/magic-link/verify`, which a custom UI may call directly. diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index 1aeaadbb0a..166d8dbf9d 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -539,6 +539,13 @@ "toMajor": 17, "rationale": "The authored `enable.apiMethods` enum is now exactly the six primitives. The eight legacy values — `upsert`, `aggregate`, `history`, `search`, `restore`, `purge`, `import`, `export` — are no longer authorable, because they are DERIVED effective operations resolved by the server's single derivation table, and an enum that lets an author name both a primitive and something derived from it has two spellings for one fact. The FROM → TO is a table rather than a rename: `upsert` → `create` + `update`; `import` → `create` + `update`; `export`, `aggregate` and `search` → `list`; `history` → `get`; and `restore` / `purge` map to NOTHING — they never derived, because `enable.trash` was retired in #2377, so the value is deleted outright. That last row is why this is a semantic entry and not a mechanical conversion, and the reason is a security one: the mapping WIDENS. An allowlist naming `history` was granting read of one record's audit trail; rewritten to `get` it grants ordinary record reads, and an allowlist naming `search` becomes a grant of full `list`. A transform that applied the table silently would broaden real API permissions without anyone reading the diff, so the rewrite is delegated to the author with the widening flagged. The reporter codemod exists for exactly that shape: `node scripts/codemod/apimethods-legacy-to-primitives.mjs` scans, reports the exact replacement per site, and FLAGS the allowlists the mapping would widen so the edit stays reviewable — it reports, it does not rewrite. Stored metadata keeps parsing (permanent tolerance, narrowing only), so nothing breaks at rest; what changes is what an author may newly write. Registered by the #6350 stock reconciliation; #3543 (P2 of #3391) predates the #6148 completeness gate. ADR-0087, #3543 (backfilled #6350)." }, + { + "surface": "sys_audit_log.action — the values 'export' and 'permission_change' left the select enum declared by plugin-audit (packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts). The same two values also left the shipped list-view filters on that object: 'permission_change' from the auth_events view and 'export' from the config_changes view", + "replacement": "nothing, for either value — both are removed rather than renamed, because neither named an event this platform records. For permission changes, read the ordinary `create` / `update` rows on the permission objects themselves: a grant or binding write is an ordinary record write and the generic audit writer already ledgers it, so a second semantically-duplicate row was never minted. For `export` there is no replacement and nothing is lost: no export feature ever wrote an audit row. A consumer filtering `sys_audit_log` on either value was reading an empty result set on every deployment, and still is — what changed is that the contract no longer promises otherwise", + "migrationId": "audit-log-action-enum-retired", + "toMajor": 17, + "rationale": "Maintainer ruling 2026-08-12 (#7675), the retirement half of a two-half verdict: the cheap writers get built (#8144 login/logout, #8145 config_change) and the enum values with no feature behind them are retired. 原则记录:空 widget + 永远查不到东西的过滤器是可见产品缺陷;审计面宁窄勿谎. The defect was false compliance on a COMPLIANCE surface, which is the sharpest form of ADR-0049 declared-≠-enforced: an auditor reading the action enum believed the platform captured permission changes and data exports, and the shipped list views and dashboard widgets showed them a filter and a tile for exactly those events. Both were permanently empty. Measured by enumerating every `sys_audit_log` writer in the repo — there are exactly two: plugin-audit`s generic hook writer, whose `actionFor` maps afterInsert/Update/Delete to create/update/delete and nothing else, and plugin-auth`s admin user-import. Neither has ever emitted `export` or `permission_change`. This is an enum-VALUE retirement, so the bookkeeping differs from a key retirement in the two ways `hook-body-crypto-hash-removed`, `dataset-measure-array-string-agg-removed` and `action-global-nav-location-removed` already record: nothing lands in RETIRED_KEYS_BY_MAJOR (no authorable KEY changed) and the four surface ratchets are expected to be byte-identical (no def changed). It differs from all three in being a SEMANTIC entry rather than a D2 conversion, and the reason is that there is no source to rewrite: `sys_audit_log` is a platform-owned, append-only object whose every field is `readonly: true`. Nobody authors an audit row and nobody authors this enum — the values appear only in rows the runtime writes and in queries consumers send. A conversion rewrites authored metadata or a stored `sys_metadata` row; this surface is neither, so the disposition is the one `BatchOptions.validateOnly` and the notification cursor already take in this major. ⚠️ Historical ROWS are deliberately untouched. A deployment that somehow holds a row with either value keeps it, and keeps reading it back: the enum is not enforced on this object at all (`validateRecord` skips `readonly` fields, and every field here is readonly), so nothing rejects stored history and no backfill is required or wanted. Deleting audit history to satisfy a schema narrowing would be the one genuinely destructive reading of this change. ADR-0049 / ADR-0087, #8147." + }, { "surface": "api.authConfig.features.passkeys / api.authConfig.features.magicLink", "replacement": "(removed — no replacement flag; the capabilities are not advertised)", @@ -1528,6 +1535,13 @@ "toMajor": 17, "rationale": "The authored `enable.apiMethods` enum is now exactly the six primitives. The eight legacy values — `upsert`, `aggregate`, `history`, `search`, `restore`, `purge`, `import`, `export` — are no longer authorable, because they are DERIVED effective operations resolved by the server's single derivation table, and an enum that lets an author name both a primitive and something derived from it has two spellings for one fact. The FROM → TO is a table rather than a rename: `upsert` → `create` + `update`; `import` → `create` + `update`; `export`, `aggregate` and `search` → `list`; `history` → `get`; and `restore` / `purge` map to NOTHING — they never derived, because `enable.trash` was retired in #2377, so the value is deleted outright. That last row is why this is a semantic entry and not a mechanical conversion, and the reason is a security one: the mapping WIDENS. An allowlist naming `history` was granting read of one record's audit trail; rewritten to `get` it grants ordinary record reads, and an allowlist naming `search` becomes a grant of full `list`. A transform that applied the table silently would broaden real API permissions without anyone reading the diff, so the rewrite is delegated to the author with the widening flagged. The reporter codemod exists for exactly that shape: `node scripts/codemod/apimethods-legacy-to-primitives.mjs` scans, reports the exact replacement per site, and FLAGS the allowlists the mapping would widen so the edit stays reviewable — it reports, it does not rewrite. Stored metadata keeps parsing (permanent tolerance, narrowing only), so nothing breaks at rest; what changes is what an author may newly write. Registered by the #6350 stock reconciliation; #3543 (P2 of #3391) predates the #6148 completeness gate. ADR-0087, #3543 (backfilled #6350)." }, + { + "surface": "sys_audit_log.action — the values 'export' and 'permission_change' left the select enum declared by plugin-audit (packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts). The same two values also left the shipped list-view filters on that object: 'permission_change' from the auth_events view and 'export' from the config_changes view", + "replacement": "nothing, for either value — both are removed rather than renamed, because neither named an event this platform records. For permission changes, read the ordinary `create` / `update` rows on the permission objects themselves: a grant or binding write is an ordinary record write and the generic audit writer already ledgers it, so a second semantically-duplicate row was never minted. For `export` there is no replacement and nothing is lost: no export feature ever wrote an audit row. A consumer filtering `sys_audit_log` on either value was reading an empty result set on every deployment, and still is — what changed is that the contract no longer promises otherwise", + "migrationId": "audit-log-action-enum-retired", + "toMajor": 17, + "rationale": "Maintainer ruling 2026-08-12 (#7675), the retirement half of a two-half verdict: the cheap writers get built (#8144 login/logout, #8145 config_change) and the enum values with no feature behind them are retired. 原则记录:空 widget + 永远查不到东西的过滤器是可见产品缺陷;审计面宁窄勿谎. The defect was false compliance on a COMPLIANCE surface, which is the sharpest form of ADR-0049 declared-≠-enforced: an auditor reading the action enum believed the platform captured permission changes and data exports, and the shipped list views and dashboard widgets showed them a filter and a tile for exactly those events. Both were permanently empty. Measured by enumerating every `sys_audit_log` writer in the repo — there are exactly two: plugin-audit`s generic hook writer, whose `actionFor` maps afterInsert/Update/Delete to create/update/delete and nothing else, and plugin-auth`s admin user-import. Neither has ever emitted `export` or `permission_change`. This is an enum-VALUE retirement, so the bookkeeping differs from a key retirement in the two ways `hook-body-crypto-hash-removed`, `dataset-measure-array-string-agg-removed` and `action-global-nav-location-removed` already record: nothing lands in RETIRED_KEYS_BY_MAJOR (no authorable KEY changed) and the four surface ratchets are expected to be byte-identical (no def changed). It differs from all three in being a SEMANTIC entry rather than a D2 conversion, and the reason is that there is no source to rewrite: `sys_audit_log` is a platform-owned, append-only object whose every field is `readonly: true`. Nobody authors an audit row and nobody authors this enum — the values appear only in rows the runtime writes and in queries consumers send. A conversion rewrites authored metadata or a stored `sys_metadata` row; this surface is neither, so the disposition is the one `BatchOptions.validateOnly` and the notification cursor already take in this major. ⚠️ Historical ROWS are deliberately untouched. A deployment that somehow holds a row with either value keeps it, and keeps reading it back: the enum is not enforced on this object at all (`validateRecord` skips `readonly` fields, and every field here is readonly), so nothing rejects stored history and no backfill is required or wanted. Deleting audit history to satisfy a schema narrowing would be the one genuinely destructive reading of this change. ADR-0049 / ADR-0087, #8147." + }, { "surface": "api.authConfig.features.passkeys / api.authConfig.features.magicLink", "replacement": "(removed — no replacement flag; the capabilities are not advertised)",