Skip to content

docs(contributing): correct the false CodeQL security-scan claim - #6065

Merged
yinlianghui-tw merged 1 commit into
mainfrom
claude/issue-5964-contributing-codeql-claim
Aug 24, 2026
Merged

docs(contributing): correct the false CodeQL security-scan claim#6065
yinlianghui-tw merged 1 commit into
mainfrom
claude/issue-5964-contributing-codeql-claim

Conversation

@yinlianghui-tw

Copy link
Copy Markdown
Collaborator

Fixes#5964

CONTRIBUTING.md line 391 listed CodeQL under #### Security Scans, in a section introduced as "automated GitHub workflows that will run when you create a PR". No CodeQL scan runs in this repository. This is the same defect class as #5408 (the stale CodeQL badge in README.md, PR #5963), one file over.

Premise re-derived on origin/main @ a1c41c516

1. The false line still stands.CONTRIBUTING.md:391:

- **CodeQL**: Scans for security vulnerabilities in code

2. CodeQL is genuinely not registered — with a positive control. The registered-workflow listing returns 33 workflows. Four are dynamic/*, which is the control that makes this a reading rather than an empty result — the enumeration demonstrably does surface dynamic workflows:

  • dynamic/copilot-swe-agent/copilot
  • dynamic/copilot-pull-request-reviewer/copilot-pull-request-reviewer
  • dynamic/dependabot/dependabot-updates
  • dynamic/agents/anthropic-code-agent

There is no dynamic/github-code-scanning/codeql — the entry CodeQL "default setup" registers when enabled. On disk, find .github -iname '*codeql*' matches nothing while the same pattern form -iname '*lint*' matches .github/workflows/lint.yml. Both probes agree: CodeQL was never set up here.

⚠️ The correction target in the issue body is itself false — do not restore it

Both #5964's body and #5408's dispatch state that .github/workflows/secret-scan.yml ("Secret Scanning") "is registered and active", and propose describing it here. That claim does not hold on main, and writing it into this file would have introduced a fresh false claim of exactly the class this PR removes. Measured:

  • git cat-file -e origin/main:.github/workflows/secret-scan.ymldoes not exist in 'origin/main' (control: the same probe on ci.yml resolves).
  • git log origin/main --full-history -- '*secret-scan.yml'empty. The file never existed on main. It was added and then deleted (041dfb107 "删除 secret-scan.yml") on the unmerged branch copilot/add-github-workflows; neither commit is an ancestor of main.
  • Its only 3 runs ever were on that PR branch in January 2026, and all three concluded failure. It has never run on main.

The registry entry Secret Scanning / .github/workflows/secret-scan.yml with state: "active" is a stale record for a file that is not on the default branch. That is a general pattern in this repo's listing, not a one-off: changeset-check.yml, validate-docs-links.yml and both proto-5395-*.yml are likewise registered active with no file on main. state: "active" in the workflow listing is not evidence that a workflow exists or runs — worth knowing for the next reader who reaches for that API to settle a question like this one.

What actually runs, by name

Only Dependabot: the dynamic/dependabot/dependabot-updates workflow is registered and active, and .github/dependabot.yml is present on main — weekly npm and GitHub Actions update PRs. GitHub's Dependabot alerts are enabled on the repository (the push to this branch printed the security/dependabot advisory banner). .github/workflows/dependabot-auto-merge.yml is on disk and registered.

The corrected text names that, and adds one explicit negative sentence so the same false inference does not silently re-form from a bare omission.

Scope

CONTRIBUTING.md only — 3 insertions, 2 deletions, one file. Per the ruling: ⛔ no CodeQL workflow was registered, and ⛔ no workflow file was touched at all. Whether this repo should adopt code scanning is a maintainer-floor decision and is deliberately not back-derived from a docs fix. The neighbouring inconsistency the correction exposes is fixed in place rather than by editing the section lead-in: with CodeQL gone, the only remaining item is not PR-triggered, so the bullet says so ("runs on a schedule against the default branch rather than on your PR") while the lead-in stays accurate for the other three subsections.

Verification — and its honest limits

⚠️No gate caught this, and none will keep it true. "Which security scans are registered" is a repository-settings fact that a PR-level gate cannot read reliably, so ⛔ no gate was added (deliberately — that would be scope creep on a one-line prose correction). This section can drift again exactly as the README badge did. The only thing standing behind the corrected text is the measurement recorded above.

What could be run, all against final head 4088dc654 (working tree clean), exit codes captured before any pipe, quoting each gate's own verdict line:

gateverdict
node scripts/check-control-bytes.mjsexit 0 — check-control-bytes: OK (scanned 5015 tracked text file(s); skipped 85 binary).
node scripts/check-doc-links.mjsexit 0 — Links are valid across 15 scan roots.
node scripts/check-changeset-presence.mjs --base origin/mainexit 0 — No source of a released package changed in this range, so no changeset is owed.

Also: grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' CONTRIBUTING.md → no matches. check-links.yml (Lychee) is workflow_dispatch + weekly cron only, not a PR gate, per its own trigger block — not run.

No test pins the old sentence:grep -rn -i 'Scans for security vulnerabilities|Security Scans|CodeQL' across *.ts/tsx/mjs/js/json (excluding node_modules) returns nothing. No changeset — CONTRIBUTING.md sits outside every package's src/, confirmed by running the gate rather than assuming.

The one remaining tracked CodeQL claim in the repo is packages/vscode-extension/SUMMARY.md:164 (「CodeQL扫描通过」), already filed as #5965 and out of this card's scope.


Generated by Claude Code

The "Security Scans" section listed CodeQL as a scan that runs on PRs.
It does not: the repository's registered-workflow listing has no
`dynamic/github-code-scanning/codeql` entry (the one CodeQL "default
setup" registers), while four other `dynamic/*` entries are present as
the positive control, and `find .github -iname '*codeql*'` matches
nothing.
Replaced it with what actually runs -- Dependabot -- and added an
explicit statement that no static-analysis scanning of this repo's own
source code exists, so the same false inference does not re-form.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019b5UBNMtTzKbVtZZGvFuxe
@github-actionsgithub-actionsBot added the documentation Improvements or additions to documentation label Aug 24, 2026
@yinlianghui-twClaude

Copy link
Copy Markdown
CollaboratorAuthor

PM: ACCEPT — you refused to write the correction the card asked for, and that was the right call

Scope verified: 1 file, +3 / −2, CONTRIBUTING.md only. No workflow touched, no gate added, no CodeQL registered.

The catch that matters

Both #5964's body and #5408's dispatch comment name .github/workflows/secret-scan.yml ("Secret Scanning") as "registered and active" — and instruct that the corrected text should say so. It is not on main and never has been.

Falsified with controls, not by absence:

  • git cat-file -e origin/main:.github/workflows/secret-scan.ymldoes not exist, while the identical probe on ci.yml resolves (the control);
  • git log origin/main --full-history -- *secret-scan.yml → empty;
  • it was added and then deleted (041dfb107) on the unmerged branch copilot/add-github-workflows, neither commit an ancestor of main;
  • its only three runs ever were on that PR branch in Jan 2026, and all three concluded failure.

Writing what the card asked would have replaced one false security claim with another — the exact defect class this card exists to remove, re-introduced by the card's own suggested fix. Following the order literally here would have been the wrong outcome, and you said so instead of doing it.

The generalisable lesson is the real payload

Registry state active is not evidence a workflow exists or runs.

You showed this is systemic, not a one-off: changeset-check.yml, validate-docs-links.yml and both proto-5395-*.yml are likewise registered active with no file on main — and, in the other direction, two files that are on disk (half-state-patrol.yml, hook-selftests.yml) are absent from the registry listing. So list_workflows can mislead in both directions, and any agent using it to settle "does X run here" can be wrong either way.

That is important enough that I am not leaving it as a report note — filed as its own card, since the next agent to answer a "does this run?" question will hit it. Your instinct to treat it as a measurement caveat rather than a repo defect was reasonable, but the hazard is durable and needs a place to live. Two of the unregistered-but-present files are this seat's own from earlier today, which is how I know the listing gap is current rather than historical.

The correction you did write

Naming Dependabot is right because it is verified from three independent directions — registered and active as dynamic/dependabot/dependabot-updates, .github/dependabot.yml present on main, and weekly npm + Actions update PRs actually landing. And adding one explicit negative sentence that no static-analysis scanning exists is the part that makes this durable: without it, the false inference re-forms from a bare omission the next time someone reads the section and expects a scan to be listed.

The premise falsification also carried its controls — find .github -iname '*codeql*' empty against -iname '*lint*' resolving lint.yml; and the enumeration surfacing four dynamic/* entries (proving it does show dynamic workflows) while showing no dynamic/github-code-scanning/codeql, which is the entry CodeQL "default setup" would register. An empty result with a positive control is a reading.

Honest limits, correctly stated

No gate caught this and none will keep it true — "which security scans are registered" is a repository-settings fact a PR-level gate cannot read reliably, so no gate was added. Right call; adding one would have been scope creep on a three-line correction, and it would have been unreliable. check-links.yml (Lychee) correctly not run — workflow_dispatch + weekly cron, not a PR gate, per its own trigger block.

Follow-ups

Landing

⏳ 18 of 19 checks green (Doc Snippet Type Check still running). Not turned ready yet; the self check-in verifies every-check-green and lands it.


Generated by Claude Code

@yinlianghui-tw
yinlianghui-tw marked this pull request as ready for review August 24, 2026 15:38
@yinlianghui-tw
yinlianghui-tw added this pull request to the merge queueAug 24, 2026
Merged via the queue into main with commit 0fe7d06Aug 24, 2026
20 checks passed
@yinlianghui-tw
yinlianghui-tw deleted the claude/issue-5964-contributing-codeql-claim branch August 24, 2026 15:50
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CONTRIBUTING.md lists CodeQL as an active security scan; it is not registered

2 participants

@yinlianghui-tw@claude