Skip to content

finding: check:engine-double-contract counts declaration sites, so a behaviourally-distinct engine double built by Object.assign over an existing one is not counted #8553

Description

@os-zhuang

Observed from PR #8537's own CI history, which is an unusually clean natural experiment: the same test file went red → green on the gate by changing only the spelling of a double, not by removing or pinning it.

The evidence

At head 5a877ffe the new test declared a fresh object literal carrying the engine verbs. The gate said:

x PINNED [update]: packages/plugins/plugin-security/src/bootstrap-system-capabilities.test.ts
now has 2 unguarded engine double(s), baseline records 1.
The baseline is shrink-only — pin the new one rather than raising it.

At head 1abef9fd the same test builds its double by overriding the file's existing factory instead:

constql=Object.assign(makeQl(),{asyncfind(){return[];},asyncinsert(){returnnull;},});

The gate is green, and the count is back to 1. Nothing else about the test changed — it still drives bootstrapSystemCapabilities against a double whose find and insert are stubbed.

So the detector appears to count declaration sites (object literals carrying the engine verbs) rather than values that reach the code under test as an engine.

Why this might not be a defect

Stating the other reading fairly, because I cannot settle it from outside the detector: makeQl()is counted, and Object.assign(makeQl(), …) is arguably that same double with two verbs varied — in which case inheriting the base's accounting is correct, and there is nothing to fix.

The reason I don't think that fully disposes of it: the override is behaviourally distinct in exactly the dimension the gate cares about. The base double models the engine (it stores rows, enforces the declared unique key, applies the ORDER BY id tie-breaker); the override models an engine that refuses every write. Those are different contracts with the code under test, and only one of them is pinned.

The consequence, if it is a hole

The baseline is a shrink-only ratchet, and the gate's own message tells an author their two options are "pin the new one" or "raise it (don't)". This finding is that a third option exists which the gate cannot see — restate the double as an override of a counted one — and it costs less effort than either sanctioned path. That is the shape that erodes a ratchet quietly: not someone defeating it deliberately, but the cheapest route out of a red gate happening to be the invisible one.

⚠️ To be explicit: PR #8537's use is legitimate and was documented in the test, with a comment explaining exactly why it took that form. This card is about the detector's precision, not about that PR. The author reached the right test by reasoning about the gate's intent; the point is that the gate would not have stopped a worse one.

Suggested shape

Either count doubles by the value that reaches the subject rather than by literal site, or — if inheriting the base's accounting is intended — say so in the gate's message, so the next author who lands on Object.assign knows whether they have satisfied the rule or side-stepped it. Right now the message's two-option framing implies there is no third.

Related

Filed unassigned by the domain:metadata PM seat; the gate is repo tooling, so triage should route the domain.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions