Skip to content

ci: detect when Dependabot stops applying dependabot.yml - #106

Merged
bgard68 merged 2 commits into
mainfrom
ci/dependabot-config-drift-guard
Aug 31, 2026
Merged

ci: detect when Dependabot stops applying dependabot.yml#106
bgard68 merged 2 commits into
mainfrom
ci/dependabot-config-drift-guard

Conversation

@bgard68

Copy link
Copy Markdown
Owner

Adds the check that would have caught the three weeks this repository spent running a config it was not actually using.

What is and is not already covered

I probed this rather than assuming. Restoring the pre-#100 config on a throwaway branch produced a failing.github/dependabot.yml check, naming the property exactly:

The property '#/updates/0/' contains additional properties
["versioning-strategy"] outside of the schema when none are allowed

So an invalid config cannot reach main through a pull request unnoticed. That half is covered, and this guard does not duplicate it.

The gap is the state afterwards. Once a rejected config is on the default branch, Dependabot falls back to the last version that parsed and keeps running from it. The validator never re-runs, because it only fires on pull requests that modify the file — so a config broken by a direct push, or broken before that check existed, stays broken and silent indefinitely. That is exactly what happened here from 2026-08-07 to 2026-08-31.

What it does

Compares what the config declares against what Dependabot actually emits, so it tests the outcome rather than the mechanism — it does not care why the config is not being applied.

  • Fails when Dependabot opens a PR for a group the config does not declare. This is the unambiguous signal: for three weeks it kept opening dotnet-minor-patch PRs, a group deleted from the file on 08-07.
  • Reports ungrouped PRs in ecosystems whose groups catch everything. Corroborating only, never fatal — security updates legitimately bypass version-update groups and are not distinguishable from this endpoint.
  • Says INCONCLUSIVE, loudly, when there is no Dependabot output since the config changed. A green check that proved nothing is the failure mode this exists to stop repeating.

Only PRs created more than 6 hours after the config's own last commit count, so a group rename does not report its own regenerated PRs as drift — a race that actually occurred today with #101/#102.

Verification

Replayed against the real incident — the Aug 24 config versus the Aug 31 PRs:

WARN #99: ungrouped, but 'github-actions' groups everything
FATAL #97: undeclared group 'dotnet-minor-patch'
WARN #95: ungrouped, but 'github-actions' groups everything
=> 1 fatal, 4 corroborating -> exit 1

It would have fired on the first Monday after the break, instead of the fourth.

Then run against every sibling repo whose config is being applied — DevSecOpsSentinel, ToDoApp, Net10Sudoku — all exit 0 with no warnings.

That last part caught a bug in the check itself. Dependabot names groups in two title shapes:

bump the <name> group with 3 updates
bump azure/login from 3.0.1 to 3.0.2 in the <name> group

Matching only the first reported healthy repos as ungrouped and would have let a stale group name through in the second form. Both shapes are now covered, with unit tests over the real titles from five repos.

Scope

WidgetWorks only, deliberately — the guard should prove itself on a live weekly run before being copied. LotteryApp, ClaudeChessApp, ToDoApp, DevSecOpsSentinel, and Net10Sudoku are all candidates once it has.

Trigger it early with workflow_dispatch if you would rather not wait for Tuesday; expect INCONCLUSIVE until Dependabot's next run, since the config changed today.

@bgard68
bgard68 merged commit e807abd into mainAug 31, 2026
8 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@bgard68