Skip to content

dependabot-auto-merge races its own reported-checks verification and refuses every green PR #46

Description

@forkwright

Finding

dependabot-auto-merge.yml's reported-checks verification runs before slow required checks have
reported, so it concludes they are missing and refuses the merge. Every dependabot PR in an adopting
repository fails, including ones whose checks all pass moments later.

The refusal is the safe direction, so nothing merges wrongly — but auto-merge never merges anything,
which is the feature's entire purpose.

Evidence

akroasis adopted this reusable in forkwright/akroasis#448. The first two real dependabot PRs after
that both failed auto-merge / auto-merge.

Run 32471047766, on dependabot/github_actions/actions-492e2c7bf6:

10:06:09 Refreshing checks status every 30 seconds.
10:06:39 cargo audit pending
cargo deny pending
10:07:40 cargo audit pass 21s
cargo deny pass 1m11s
10:07:41 ##[error]No required verification check matching any of: gateattestation gategate.
10:07:44 Real verification checks missing/unsuccessful.

The gate had not failed. Gate Attestation run 32471047696 on that same head completed
successfully, and the PR's final check set contains gate / gate → normalises to gategate,
which is exactly the token being searched for:

SUCCESS gate / gate -> gategate
SUCCESS gate / full-gate-build -> gatefullgatebuild
SUCCESS cargo audit -> cargoaudit
SUCCESS cargo deny -> cargodeny
SUCCESS osv-scanner -> osvscanner

So the matcher is correct and the timing is not.

Why it happens

The wait step is gh pr checks "$PR_URL" --watch --interval 30 --required. --watch waits on the
checks that exist at that moment; a required check whose run has not yet created its check-run is
not among them and is not waited for. Above, only cargo audit and cargo deny had reported when
watching began, so --watch returned as soon as those two passed — while gate / gate was still
building.

require_passed_check then queries gh pr checks --json name,bucket immediately, finds no gate
entry, and takes the No required verification check matching branch, which is written for the case
this workflow exists to catch: a required check that silently never reports.

That is the irony worth naming. This reusable's distinguishing feature over branch protection is that
it notices when a real verification check fails to report at all. The bug is that it evaluates
that question at a moment when a slow check legitimately has not reported yet, so a healthy repo is
indistinguishable from the failure mode.

Gate Attestation is the check most likely to trip this everywhere, because it compiles and tests the
workspace and is therefore always the slowest — the check most worth waiting for is the one most
likely to be missed.

Desired correction

Make the wait cover the checks the verification will later demand, rather than whatever happened to
exist when watching started. Either:

  • poll require_passed_check's tokens until each is present and terminal (with a timeout that
    fails closed), rather than reading the check list once; or
  • keep --watch but re-enter it if any required token is still unreported after it returns, so
    "returned early because a check had not been created" is retried rather than treated as absence.

A bare timeout increase would not fix it — the failure is not that the wait is too short, it is that
the wait is not watching the thing the later check requires.

Done when: a dependabot PR whose checks all eventually pass reaches auto-merge success without
intervention, and a PR whose gate genuinely never reports still fails — with a fixture covering both,
since the current behaviour passes the second half already and it is the first half that regressed.

Interim, for adopters

The failing check is not a required context, so mergeStateStatus stays mergeable and a maintainer
can merge manually after confirming the required set is green. akroasis#454 and #455 were merged that
way. Nothing is unsafe; the automation is simply not doing its job.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions