chore(versions): bump scanner pins and unblock the tool version sync - #46
Open
Cre-eD wants to merge 4 commits into
Open
chore(versions): bump scanner pins and unblock the tool version sync#46Cre-eD wants to merge 4 commits into
Cre-eD wants to merge 4 commits into
Conversation
The weekly Tool Version Sync check has failed on every Dependabot bump of versions/Dockerfile since early July. Dependabot updates the FROM pins but cannot run tools/sync-tool-versions.sh, so the new pin never reaches the consumer files and the --check gate fails. Nothing merges, and consumers keep scanning with frozen detection content. Bump the pins and propagate them: trufflehog 3.95.2 -> 3.95.9 syft v1.44.0 -> v1.51.0 trivy 0.70.0 -> 0.74.0 grype v0.112.0 -> v0.117.0 semgrep 1.161.0 -> 1.174.0 Digests are the current index digests resolved from each registry. Every image was pulled and run to confirm it reports the expected version. TruffleHog deliberately stays on the 3.95.x line rather than moving to 3.97.0. 3.97.0 rewrites the Lob detector (pattern and verification semantics) and that change deserves its own evaluation rather than riding along with a routine pin refresh. To stop the deadlock recurring, add a Tool Version Autosync workflow. It is dispatch-driven: give it a branch name and it runs --apply, commits and pushes. That covers the Dependabot case without a local clone. It is not wired to pull_request_target on purpose. Dependabot-triggered pull_request workflows get a read-only token, and the usual workaround (pull_request_target plus a checkout of the PR head) is the Poisoned Pipeline Execution vector that this repo's own rules classify as severity ERROR: gha-pull-request-target-with-pr-head-checkout, gha-pull-request-target-implicit-base-ref and gha-checkout-persist-credentials-true. The repo that defines those rules should not be the repo that violates them. The push uses the http.extraheader form that rule gha-github-token-in-url recommends, so no credentials are persisted into .git/config. Also surface an actionable job summary when the gate fails, so the fix is visible on the run page instead of only in a log annotation. Signed-off-by: Dmitrii Creed <creeed22@gmail.com>
Security Scan ResultsRepository:
Scanned at 2026-08-23 11:17 UTC |
Semgrep Scan ResultsRepository:
Scanned at 2026-08-23 11:17 UTC |
The earlier pin in this branch held trufflehog on the 3.95 line to
avoid a detector change. That was backwards: the change is a
false-positive fix, and staying on 3.95.x keeps the false positive.
Up to and including 3.96.0 the Lob detector pattern was:
\b((live|test)_[a-zA-Z0-9_]{35})\b
which matches any identifier of the right shape, so an ordinary test
function name of exactly 35 characters after the `test_` prefix is
reported as a secret. Worse, a bare HTTP 403 from the verification
endpoint was treated as proof the key was live, so these land as
Verified=true rather than as a low-confidence hit.
3.97.0 narrows the pattern to hex only:
\b((live|test)_[a-f0-9]{35})\b
and only reports verified when the 403 body carries an error code that
means the key authenticated but the account was rejected.
Measured against a fixture holding one such test function name, using
the same flags trufflehog-scan/scan.sh passes:
3.95.9 1 finding, detector Lob, Verified=true
3.97.0 0 findings
fail-on-secrets defaults to true, so on the old pin that is a red
build in any consumer repo with a Python test suite. Nothing in this
repo excludes the Lob detector, so there was no mitigation in place.
Signed-off-by: Dmitrii Creed <creeed22@gmail.com>
`ref: ${{ inputs.branch }}` handed a free-text dispatch input straight to
`actions/checkout`, which resolves branches, SHAs and `refs/pull/<n>/head`,
the ref this repository holds for fork pull requests. The next step ran
`tools/sync-tool-versions.sh` out of that tree in a job with
`contents: write`, and the step after it put a token in the environment,
so fork-authored content could have prepended a fake `git` to
`$GITHUB_PATH` or planted `.git/hooks/pre-commit` and captured write
access to the repository that defines this org's composite actions. A tag
or branch write here is a supply-chain compromise of every consumer.
The header claimed a human dispatch "keeps a full-scope token without ever
running PR content, so there is no untrusted-head problem to mitigate".
That was wrong: dispatch was the way PR content got run. It is replaced
with the real threat model, and the consumer-file list is corrected from
five files to the seven stages across six files the script actually maps.
Changes:
* `Validate target branch` runs before the checkout and accepts only
`dependabot/*`, rejecting `..` and trailing slashes. Forks cannot create
branches here, so the only tree this job can execute is one Dependabot
wrote in this repository. The checkout consumes the validated step
output rather than the raw input, which also stops
`HEAD:refs/heads/${TARGET_BRANCH}` turning a `refs/heads/...` input into
`refs/heads/refs/heads/...`.
* The privileged commit runs with `core.hooksPath=/dev/null` and
`--no-verify`, so a hook planted in the checked-out tree cannot run.
* The push credential moves off `git`'s argv, where it was readable from
the runner's process table for the lifetime of the push, into
`.git/config`, written with a shell builtin and removed in an
`always()` step.
* New rule `gha-dispatch-input-ref-checkout` (ERROR) with positive and
negative fixtures. All 28 existing `gha-*` rules key on
`pull_request_target` or `workflow_run`; none matched `workflow_dispatch`
plus an input-derived checkout ref in a job holding write scope or
secrets. The rule fires twice on the pre-fix workflow and not at all on
the fixed one.
Signed-off-by: Dmitrii Creed <creeed22@gmail.com>
Two of the seven tracked stages, `actionlint` and `shellcheck`, resolve to `.github/workflows/lint.yml`, and `.github/dependabot.yml` tracks `directory: /versions` for the docker ecosystem, so Dependabot will open bumps for both. `--apply` then rewrites a file under `.github/workflows/`, and a push authenticated with `GITHUB_TOKEN` is refused: "refusing to allow a GitHub App to create or update workflow `.github/workflows/lint.yml` without `workflows` permission". That permission cannot be granted to the Actions token through `permissions:`, so the run committed and then died on a bare push error with nothing explaining why. Autosync now detects a staged path under `.github/workflows/` before committing and stops with the fix instructions. The cleaner long-term option, moving those two pins out of `lint.yml` into a non-workflow file the workflow sources, is left for a separate change. Also here: * `tools/test-sync-tool-versions.sh` copies the tracked tree into a scratch directory, applies a synthetic Dockerfile bump per stage group, and asserts exactly which files `--apply` rewrites. The actionlint case is the one that surfaces the constraint above; nothing was asserting the file set before, which is why it went unnoticed. Wired into the `Tool Version Sync` job. * The embedded `python3 -c` in the propagation loop becomes a `python3 - <<'PY'` block, matching the style already used in `semgrep-scan/run-tests.sh`. * README documents the autosync workflow, its `dependabot/*` allowlist, the GITHUB_TOKEN re-run caveat and the workflow-file exception. It previously covered only the manual `--apply` flow. Signed-off-by: Dmitrii Creed <creeed22@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Tool Version Synchas failed on every Dependabot bump ofversions/Dockerfilesince early July: six consecutive runs (07-06, 07-13, 07-20, 07-27, 08-10, 08-17), all ondependabot/docker/versions/**branches.The cause is structural, not a flake. Dependabot can bump the
FROMpins inversions/Dockerfile, but it cannot runtools/sync-tool-versions.sh, so the new pin never reaches the five consumer files.--checkthen fails and the bump cannot merge. Because that gate blocks every scanner bump, consumers of these composite actions have been scanning with detection content frozen since roughly mid-April.This PR does the bump, propagates it, and removes the deadlock.
Pins
Latest versions were checked against upstream releases, and each digest is the current index digest resolved from the registry (
docker buildx imagetools inspect), not a per-platform manifest digest. Every image below was pulled and executed to confirm it reports the expected version.sha256:ff4c95e9df7d645daf2140e3ca1039031c63106268d5fbb25feb43ceca1bcc33sha256:678bfa565b60f747aac0f8e964fe5588a24445b8d0a480e91f6efd70020dfbb0sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969sha256:ddf9e9f204049f3a4a0955ef70873cabab6a31432125ad4f20a490b54950a253sha256:f1f7b71861c7b28b6e0f661225a2c4f58a484f5d0f182465c6d6b3b22f972adeTrivy had no open bump PR at all, so it is included here.
actionlintandshellcheckare untouched; they were already current.Why the TruffleHog bump matters beyond freshness
This one is a false-positive fix, not just a version increment.
Up to and including 3.96.0, the Lob detector pattern was:
That matches any identifier of the right shape, not just a key. An ordinary test function name with exactly 35 characters after the
test_prefix matches it. On top of that, a bare HTTP 403 from the verification endpoint was treated as proof the key was live, so these surface asVerified=truerather than as a low-confidence hit.3.97.0 narrows the pattern to hex only:
and only reports verified when the 403 body carries an error code meaning the key authenticated but the account was rejected.
Measured, not assumed. A fixture tree containing a single Python file with
def test_calculate_total_price_with_tax_rate():(exactly 35 characters aftertest_), scanned with the same flagstrufflehog-scan/scan.shpasses (filesystem --json --no-update --exclude-detectors=FormBucket):DetectorName: Lob,Verified: true, rawtest_calculate_total_price_with_tax_ratefail-on-secretsdefaults to true, so on any pin below 3.97.0 that is a red build in every consumer repo that has a Python test suite with a function name of that length. Nothing in this repo passes--exclude-detectors=Lob(the only exclusion isFormBucket), so there was no mitigation in place.Making the sync self-healing
New workflow:
.github/workflows/tool-version-autosync.yml. Give it a branch name from the Actions tab; it runs--apply, commits and pushes. That unblocks a Dependabot PR without anyone cloning the branch locally.It is not wired to
pull_request_target, and that is a deliberate call rather than an oversight:pull_requestworkflows receive a read-onlyGITHUB_TOKENand cannot push.pull_request_targetplus a checkout of the PR head, is the Poisoned Pipeline Execution vector. This repo's own ruleset classifies it as severity ERROR in three separate rules:gha-pull-request-target-with-pr-head-checkout,gha-pull-request-target-implicit-base-ref, andgha-checkout-persist-credentials-true.An earlier draft of this change used exactly that pattern. Running
semgrep-scan/rules/against it produced three ERROR findings in a repo that otherwise has zero. The repo that defines those rules should not be the repo that violates them, so the design was changed.Correction: the first version of this workflow was still vulnerable
An earlier revision of this section claimed that "the dispatch trigger keeps a full-scope token without ever running PR-authored content, so there is no untrusted head to mitigate in the first place", and the workflow header said the same. That was wrong, and it has been fixed rather than softened. Dispatch was precisely how PR content would have got run.
ref: ${{ inputs.branch }}handed a free-text dispatch input straight toactions/checkout. That resolves branches, SHAs andrefs/pull/<n>/headalike, and the base repository holdsrefs/pull/<n>/headfor fork pull requests. The next step rantools/sync-tool-versions.shout of that tree, in a job withcontents: write, and the step after it put a token in the environment. Attacker-authored content in the checked-out tree needed only to prepend a fakegitto$GITHUB_PATH, or plant.git/hooks/pre-commit, to capture write access to the repository that defines this org's composite actions. A tag or branch write here is a supply-chain compromise of every consumer, so "only maintainers can dispatch" is not a sufficient control: the maintainer supplies the ref, and the ref decides whose code runs.What changed:
Validate target branchstep runs before the checkout and accepts onlydependabot/*, rejecting..and trailing slashes. Forks cannot create branches in this repository, so the only tree the job can ever execute is one Dependabot wrote here. The checkout consumes that step's validated output rather than the raw input.push origin "HEAD:refs/heads/${TARGET_BRANCH}"turned arefs/heads/fooinput intorefs/heads/refs/heads/foo. Confirmed: only shortdependabot/*names now reach the push.git -c core.hooksPath=/dev/null commit --no-verify, so a hook planted in the checked-out tree cannot run even if the allowlist is ever widened.git's argv.git -c http.extraheader=...puts the token in the runner's process table for the whole lifetime of the push. It is now written into.git/configwith a shell builtin (so it is never a process argument either) and removed in analways()step. Every checkout still keepspersist-credentials: false.The repo's own ruleset could not have caught this, which is the part worth fixing beyond this one file. All 28 existing
gha-*rules key onpull_request_targetorworkflow_run; nothing matchedworkflow_dispatchplus an input-derived checkout ref. So this PR adds the rule for the class:gha-dispatch-input-ref-checkout(severity ERROR) insemgrep-scan/rules/github-actions.yml, matchingworkflow_dispatch:plusactions/checkoutwithref:derived frominputs.*/github.event.inputs.*in a job carryingcontents: writeor asecrets.reference. Both orderings of the privileged marker relative to the checkout are covered.semgrep-scan/tests/.github/workflows/cases.yml, including the remediated shape (ref:from a validated step output) as an# ok:case.Known limitation, documented in the workflow header: commits pushed with
GITHUB_TOKENdo not start new workflow runs, soTool Version Syncwill not re-run by itself on the pushed commit. The autosync run verifies the propagation instead, since--applyre-runs--checkinternally and exits non-zero if drift survives. To get a green gate on the PR afterwards, close and reopen it or comment@dependabot rebase. Making this fully automatic would need a GitHub App token; that is a separate decision and is called out in the workflow comments.Also added: an actionable job summary on gate failure, so the exact fix command appears on the run page rather than only in a log annotation.
The autosync workflow could never have completed for two of the seven tracked tools
tools/sync-tool-versions.shroutes theactionlintandshellcheckstages to.github/workflows/lint.yml, and.github/dependabot.ymltracksdirectory: /versionsfor the docker ecosystem, so Dependabot will open bumps for both.--applythen rewrites a file under.github/workflows/, and a push authenticated withGITHUB_TOKENis refused:That permission cannot be granted to the Actions token through
permissions:at all. The run would have committed and then died on a bare push error with nothing explaining why.Autosync now detects a staged path under
.github/workflows/before committing and stops with the fix instructions. Those two stages have to be propagated locally and pushed with a user credential.The cleaner long-term fix, not done here: move the actionlint and shellcheck pins out of
lint.ymlinto a non-workflow file (for exampleversions/pins.env) that the workflow sources. Then no bump of any tracked tool touches.github/workflows/**and autosync covers all seven stages. That is a change to howlint.ymlgets its versions and deserves its own PR.Why this went unnoticed: nothing asserted which files a bump rewrites
New
tools/test-sync-tool-versions.sh. It copies the tracked tree into a scratch directory, applies a synthetic Dockerfile bump per stage group, and asserts exactly which files--applyrewrites, that the new pin reached each of them, that--checkfails before and passes after, and that nothing else changed. Two cases:actionlint(expects.github/workflows/lint.yml, the case that surfaces the constraint above) andsemgrep(expectssemgrep-scan/action.ymlandsemgrep-scan/run-tests.sh).Wired into the
Tool Version Syncjob. Confirmed non-vacuous: pointing the actionlint case at the wrong expected file makes it fail with a diff of want vs got.Other fixes in this round
.github/workflows/lint.ymlwas the one missing, which is the same omission that produced the failure above. Corrected in the header, and the constraint is now documented in the script header too.python3 -cheredoc in the propagation loop is now apython3 - <<'PY'block, matching the style already used insemgrep-scan/run-tests.sh.README.mddocumented only the manual--applyflow. It now covers the autosync workflow, itsdependabot/*allowlist and why the allowlist exists, theGITHUB_TOKENre-run caveat, the workflow-file exception, and the new test script.Suggested split, not performed
This PR now carries two things with very different blast radii, and I would rather ask than decide unilaterally:
tool-version-sync.ymlstep summary. Reverting is a one-commitgit revert.contents: writeon the repo that defines the org's composite actions, and it is the piece that had the P0.Splitting them would let the pin bumps land now, unblocking scanner freshness for every consumer, while the autosync workflow gets its own review. The new semgrep rule and its fixtures could go with either half, though they are most useful alongside the workflow.
Say the word and I will split it. I have not split it unprompted because the branch is already reviewed as a unit and re-splitting invalidates that review.
Supersedes
#45 targets TruffleHog 3.96.0 and should be closed in favour of this PR. 3.96.0 still carries the old
[a-zA-Z0-9_]{35}Lob pattern, so it does not resolve the false positive described above; this PR goes to 3.97.0, which does. It also covers Syft, Trivy, Grype and Semgrep, and it propagates the pins so the sync gate actually passes, which #45 cannot do on its own.Verification
All run locally against this branch:
tools/sync-tool-versions.sh --checkpasses (All tool versions in sync with versions/Dockerfile).tools/test-sync-tool-versions.shpasses both cases. Negative control run: it fails as expected when given a wrong expected-file set.actionlint(pinned 1.7.12 image, the same invocation aslint.yml) exits 0.shellcheck(pinned v0.11.0 image,--severity=warning, same file list aslint.yml) exits 0, and the new test script is also clean at--severity=style.semgrep-scan/run-tests.shpasses: 58/58github-actions.ymlfixture cases (up from 56, the two new ones being the rule-20 positive and negative), all other rule files unchanged, and the full-repo scan reports 0 findings including the fixed autosync workflow.action.ymlanddependabot.ymlparses as YAML (37 files, 0 failures).