Skip to content

An UNREACHABLE federated remote is reported as missing_table — a transient outage is indistinguishable from a dropped table, and aborts boot under the default onMismatch: 'fail' #11166

Description

@os-elon

Found while implementing #10961 (out of scope there: that card scopes which datasources a background tick introspects, not how a failed introspection is classified).

What is true at head

ExternalDatasourceService.validateEach (packages/services/service-datasource/src/external-datasource-service.ts) turns a per-object throw into a row rather than rejecting the whole report — deliberately, so one bad object does not erase the verdicts of the others:

this.validateObject(o.name).catch((err): SchemaValidationResult=>({ok: false,datasource: o.datasource??'default',object: o.name,diffs: [{kind: 'missing_table',remoteName: ...,actual: err.message,severity: 'error'}],}))

The row it invents is kind: 'missing_table', severity: 'error'the same shape a genuinely dropped remote table produces. But validateObject throws for reasons that have nothing to do with the remote schema: the first thing it does after resolving the datasource is await this.config.introspect(datasource), so a refused connection, a DNS failure, an auth expiry or a timeout all land here.

Measured against the real service at 724b7316d3 (introspector throwing connect ECONNREFUSED 10.0.0.5:5432, one federated object on a reachable-in-metadata datasource):

{ "ok": false, "results": [ { "ok": false, "datasource": "wh_a", "object": "wh_a_orders",
"diffs": [ { "kind": "missing_table", "remoteName": "orders",
"actual": "connect ECONNREFUSED 10.0.0.5:5432", "severity": "error" } ] } ] }

The connection error survives only as free text in actual. Nothing in kind, severity or ok separates "the network was down for 30 seconds" from "someone dropped the table".

Why that matters at the two consumers

  • Boot gateExternalValidationPlugin.runValidation (packages/runtime) applies the datasource's external.validation.onMismatch, whose default is fail, and throws ExternalSchemaMismatchError, which aborts boot (ADR-0015 §5.2). So a transient remote outage during startup is currently a refusal to start, reported as a schema mismatch. That is the shape AGENTS.md warns about in the announceAllClear note right above it: converting a transient dependency outage into a new functional failure mode.
  • Background drift checkerExternalValidationPlugin.runDriftCheck emits one external.schema.drift event per !r.ok row, consumed by audit / notification. A remote that is briefly unreachable therefore raises drift alerts (and audit rows) claiming its schema changed, on every tick it stays down.

Not obviously a one-line fix

The row has to keep something non-ok — silently reporting ok: true for an object nobody could validate would be the "invent an answer" failure the durability read-seam rule names. The open questions are for triage, not for this filing:

  • does SchemaDiffEntry (packages/spec) need a distinct kind for "could not be read", or does the report need an unreachable/undetermined channel beside results? — a spec-surface decision either way;
  • should onMismatch apply at all to an object whose remote could not be reached, or is that a separate policy;
  • and the discrimination itself (is this error a connection failure or a schema fact?) has a precedent in the repo's READ_FAILURE_DISCRIMINATORS shape rather than a hand-rolled err.code test.

Landing point is packages/services/service-datasource for the classification, with a consumer decision in packages/runtime — filed rather than folded into #10961, whose diff is runDriftCheck's call scoping.

Unassigned — filed for triage to level and route. Back-link: #10961, sibling #10962.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions