ci: detect when Dependabot stops applying dependabot.yml - #106
Merged
Conversation
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the check that would have caught the three weeks this repository spent running a config it was not actually using.
What is and is not already covered
I probed this rather than assuming. Restoring the pre-#100 config on a throwaway branch produced a failing
.github/dependabot.ymlcheck, naming the property exactly:So an invalid config cannot reach
mainthrough a pull request unnoticed. That half is covered, and this guard does not duplicate it.The gap is the state afterwards. Once a rejected config is on the default branch, Dependabot falls back to the last version that parsed and keeps running from it. The validator never re-runs, because it only fires on pull requests that modify the file — so a config broken by a direct push, or broken before that check existed, stays broken and silent indefinitely. That is exactly what happened here from 2026-08-07 to 2026-08-31.
What it does
Compares what the config declares against what Dependabot actually emits, so it tests the outcome rather than the mechanism — it does not care why the config is not being applied.
dotnet-minor-patchPRs, a group deleted from the file on 08-07.Only PRs created more than 6 hours after the config's own last commit count, so a group rename does not report its own regenerated PRs as drift — a race that actually occurred today with #101/#102.
Verification
Replayed against the real incident — the Aug 24 config versus the Aug 31 PRs:
It would have fired on the first Monday after the break, instead of the fourth.
Then run against every sibling repo whose config is being applied — DevSecOpsSentinel, ToDoApp, Net10Sudoku — all exit 0 with no warnings.
That last part caught a bug in the check itself. Dependabot names groups in two title shapes:
Matching only the first reported healthy repos as ungrouped and would have let a stale group name through in the second form. Both shapes are now covered, with unit tests over the real titles from five repos.
Scope
WidgetWorks only, deliberately — the guard should prove itself on a live weekly run before being copied. LotteryApp, ClaudeChessApp, ToDoApp, DevSecOpsSentinel, and Net10Sudoku are all candidates once it has.
Trigger it early with
workflow_dispatchif you would rather not wait for Tuesday; expect INCONCLUSIVE until Dependabot's next run, since the config changed today.