Found while implementing #10961 (out of scope there: that card scopes which datasources a background tick introspects, not how a failed introspection is classified).
What is true at head
ExternalDatasourceService.validateEach (packages/services/service-datasource/src/external-datasource-service.ts) turns a per-object throw into a row rather than rejecting the whole report — deliberately, so one bad object does not erase the verdicts of the others:
this.validateObject(o.name).catch((err): SchemaValidationResult=>({ok: false,datasource: o.datasource??'default',object: o.name,diffs: [{kind: 'missing_table',remoteName: ...,actual: err.message,severity: 'error'}],}))The row it invents is kind: 'missing_table', severity: 'error' — the same shape a genuinely dropped remote table produces. But validateObject throws for reasons that have nothing to do with the remote schema: the first thing it does after resolving the datasource is await this.config.introspect(datasource), so a refused connection, a DNS failure, an auth expiry or a timeout all land here.
Measured against the real service at 724b7316d3 (introspector throwing connect ECONNREFUSED 10.0.0.5:5432, one federated object on a reachable-in-metadata datasource):
{ "ok": false, "results": [ { "ok": false, "datasource": "wh_a", "object": "wh_a_orders",
"diffs": [ { "kind": "missing_table", "remoteName": "orders",
"actual": "connect ECONNREFUSED 10.0.0.5:5432", "severity": "error" } ] } ] }The connection error survives only as free text in actual. Nothing in kind, severity or ok separates "the network was down for 30 seconds" from "someone dropped the table".
Why that matters at the two consumers
- Boot gate —
ExternalValidationPlugin.runValidation (packages/runtime) applies the datasource's external.validation.onMismatch, whose default is fail, and throws ExternalSchemaMismatchError, which aborts boot (ADR-0015 §5.2). So a transient remote outage during startup is currently a refusal to start, reported as a schema mismatch. That is the shape AGENTS.md warns about in the announceAllClear note right above it: converting a transient dependency outage into a new functional failure mode. - Background drift checker —
ExternalValidationPlugin.runDriftCheck emits one external.schema.drift event per !r.ok row, consumed by audit / notification. A remote that is briefly unreachable therefore raises drift alerts (and audit rows) claiming its schema changed, on every tick it stays down.
Not obviously a one-line fix
The row has to keep something non-ok — silently reporting ok: true for an object nobody could validate would be the "invent an answer" failure the durability read-seam rule names. The open questions are for triage, not for this filing:
- does
SchemaDiffEntry (packages/spec) need a distinct kind for "could not be read", or does the report need an unreachable/undetermined channel beside results? — a spec-surface decision either way; - should
onMismatch apply at all to an object whose remote could not be reached, or is that a separate policy; - and the discrimination itself (is this error a connection failure or a schema fact?) has a precedent in the repo's
READ_FAILURE_DISCRIMINATORS shape rather than a hand-rolled err.code test.
Landing point is packages/services/service-datasource for the classification, with a consumer decision in packages/runtime — filed rather than folded into #10961, whose diff is runDriftCheck's call scoping.
Unassigned — filed for triage to level and route. Back-link: #10961, sibling #10962.
Found while implementing #10961 (out of scope there: that card scopes which datasources a background tick introspects, not how a failed introspection is classified).
What is true at head
ExternalDatasourceService.validateEach(packages/services/service-datasource/src/external-datasource-service.ts) turns a per-object throw into a row rather than rejecting the whole report — deliberately, so one bad object does not erase the verdicts of the others:The row it invents is
kind: 'missing_table',severity: 'error'— the same shape a genuinely dropped remote table produces. ButvalidateObjectthrows for reasons that have nothing to do with the remote schema: the first thing it does after resolving the datasource isawait this.config.introspect(datasource), so a refused connection, a DNS failure, an auth expiry or a timeout all land here.Measured against the real service at
724b7316d3(introspector throwingconnect ECONNREFUSED 10.0.0.5:5432, one federated object on a reachable-in-metadata datasource):{ "ok": false, "results": [ { "ok": false, "datasource": "wh_a", "object": "wh_a_orders", "diffs": [ { "kind": "missing_table", "remoteName": "orders", "actual": "connect ECONNREFUSED 10.0.0.5:5432", "severity": "error" } ] } ] }The connection error survives only as free text in
actual. Nothing inkind,severityorokseparates "the network was down for 30 seconds" from "someone dropped the table".Why that matters at the two consumers
ExternalValidationPlugin.runValidation(packages/runtime) applies the datasource'sexternal.validation.onMismatch, whose default isfail, and throwsExternalSchemaMismatchError, which aborts boot (ADR-0015 §5.2). So a transient remote outage during startup is currently a refusal to start, reported as a schema mismatch. That is the shape AGENTS.md warns about in theannounceAllClearnote right above it: converting a transient dependency outage into a new functional failure mode.ExternalValidationPlugin.runDriftCheckemits oneexternal.schema.driftevent per!r.okrow, consumed byaudit/notification. A remote that is briefly unreachable therefore raises drift alerts (and audit rows) claiming its schema changed, on every tick it stays down.Not obviously a one-line fix
The row has to keep something non-
ok— silently reportingok: truefor an object nobody could validate would be the "invent an answer" failure the durability read-seam rule names. The open questions are for triage, not for this filing:SchemaDiffEntry(packages/spec) need a distinctkindfor "could not be read", or does the report need an unreachable/undetermined channel besideresults? — a spec-surface decision either way;onMismatchapply at all to an object whose remote could not be reached, or is that a separate policy;READ_FAILURE_DISCRIMINATORSshape rather than a hand-rollederr.codetest.Landing point is
packages/services/service-datasourcefor the classification, with a consumer decision inpackages/runtime— filed rather than folded into #10961, whose diff isrunDriftCheck's call scoping.Unassigned — filed for triage to level and route. Back-link: #10961, sibling #10962.