Skip to content

Publication guard: fail the build when published research artifacts carry paths, credentials or scratchpad references #554

Description

@stranske

Why

research-program/artifacts/work-bundle/ is prepared to be handed to colleagues in the owner's work environment, and the rest of research-program/ is published to a public repository. Two disclosure-class defects have already reached this tree, both invisible on a casual read:

  • A credential scan printed token prefixes into a working file before it was caught by hand.
  • A batch of issue bodies carried a drafting agent's private working-directory paths (clones/<Repo>/...), which made twenty-two issues unactionable across the fleet and had to be repaired one by one.

Both were found by a person looking. Neither would have been caught by any check in this repository, because there is none. The bundle's own redaction pass is a one-time human-reviewed step, not a gate, so nothing prevents the next artifact from reintroducing what the last one removed.

Evidence: research-program/artifacts/work-bundle/REDACTIONS.md records the manual redaction pass and its scope reduction; README.md states the disclosure standard this issue enforces; research-program/CHECKPOINT.md records the repair of the path-citation defect.

Scope

A publication guard: one script plus a CI job that fails when a file under research-program/ contains material that must not be published.

Non-Goals

  • No scanning outside research-program/; ordinary repository code is out of scope here.
  • No attempt to detect proprietary manager content by meaning. This guard catches mechanical patterns only, and must not pretend otherwise.
  • No auto-redaction. A hit fails the build and names the file and line; a human or the engine decides what to do.

Tasks

  • Add scripts/check_publication_safety.py scanning every file under research-program/ for: an absolute home path (/Users/), a credential prefix (sk-ant-, sk-proj-, ghp_, github_pat_, lsv2_, crsr_, AIza), a private-key header (BEGIN RSA PRIVATE KEY, BEGIN OPENSSH PRIVATE KEY), a scratchpad or drafting-agent path (clones/, /private/tmp/, scratchpad/), and an internal hostname pattern (.local: or a bare localhost: port reference).
  • Report every hit in one pass with file, line number and which rule matched; do not stop at the first.
  • Support an explicit allowlist file, research-program/.publication-allow, holding one path:rule pair per line with a required trailing comment giving the reason, so a deliberate exception is visible and justified rather than a silent skip.
  • Exit non-zero on any unallowed hit; print a summary line naming the counts for every rule including the zeros, so a clean run is distinguishable from a run that scanned nothing.
  • Print the number of files scanned, so a guard that matched nothing because it walked the wrong directory is visible.
  • Add .github/workflows/publication-guard.yml running the script on pull requests and on pushes to the default branch.
  • Add tests/test_publication_safety.py with fixtures for each rule, an allowlisted exception, and a clean tree.

Acceptance Criteria

  • python scripts/check_publication_safety.py exits non-zero on a fixture containing a token prefix and names the file, the line and the rule.
  • The same fixture allowlisted with a reason exits zero, and an allowlist entry without a reason is itself an error.
  • A clean tree exits zero and prints the per-rule counts including zeros and the number of files scanned.
  • Pointing the scanner at an empty directory fails rather than passing, because zero files scanned is not evidence of safety.
  • The workflow runs on pull requests and blocks merge on failure.

Test Gate

tests/test_publication_safety.py, run by this repo's CI gate. Demonstrate the gate by deliberately weakening it: drop the ghp_ pattern from the rule set, confirm the corresponding fixture test fails, then revert and confirm it passes. Record both outcomes in the pull request.

Implementation Notes

This is a gate, so it owes the gate questions. What clears it: removing the offending text, or allowlisting it with a stated reason, both of which are possible while the gate is closed. What it measures and what it drains are the same set of files. What it prints when fully satisfied: every rule with a zero count plus the files-scanned total, so a satisfied gate and a broken gate never look alike. Keep it dependency-free so it can run anywhere, including in the target work environment where installing packages is not assumed.

Verification follow-up (adversarial review, 2026-09-14)

An independent review of the implementation PR found the work incomplete while every box was ticked. These are the remaining tasks.

  • Match the full PEM private-key family in BOTH the scanner and the exporter, including -----BEGIN PRIVATE KEY----- (PKCS#8) and -----BEGIN ENCRYPTED PRIVATE KEY-----. PKCS#8 currently scans clean, and it is the default output of modern tooling.
  • Define the header set once and have both scripts consume it, so the two copies of a security rule cannot drift.
  • One test per key format, each asserting the scan FAILS on it.
  • Wire prepare_publication.py into publication-guard.yml, or drop the claim that preparation is integrated; the workflow currently runs only the checker.
  • Make the "clean preserved" tests assert that files were actually examined, not merely that nothing was redacted — they pass today against a no-op returning files_redacted=0.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:highOpener picks these first within priority tier

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions