Fix agent-help hidden-flag leak; make check-surface a real drift gate by jeremy · Pull Request #542 · basecamp/basecamp-cli

jeremy · 2026-07-18T06:35:49Z

Closes#539.

TL;DR

#539 asked to reconcile the committed .surface snapshot with scripts/check-cli-surface.sh, on the premise that the file had drifted from its generator and CI didn't catch it. Investigating, the premise doesn't hold: .surface is owned and enforced by a different generator, which it already matches exactly. So instead of regenerating the file, this PR fixes the two things that are actually wrong and turns the misleading check-surface target into a real gate.

What's actually going on

There are two independent surface generators, and they disagree:

	`scripts/check-cli-surface.sh` (shell)	`surface.SnapshotString` (Go, `github.com/basecamp/cli`)
Walks	`--help --agent` JSON	the Cobra command tree
Globals per subcommand	curated → salient only (post-#499)	full inherited globals
Hidden flags (`--assignee`)	emitted	excluded
`--version` leak (#368)	had it pre-#499	never had it (`--version` is local, not inherited)
Output	~7.6k lines	~16.6k lines
Enforced by	nothing — `check-surface` just echoed a `/tmp` line count	`TestSurfaceSnapshot` (`internal/commands/surface_test.go`)

The committed .surface is byte-identical (modulo trailing newline) to the Go generator's output. TestSurfaceSnapshot passes precisely because .surface == surface.SnapshotString(root), and that test is the CI catch #539 thought was missing. Cross-version compatibility is separately handled by check-surface-compat → check-cli-surface-diff.sh, entirely on the script side.

Regenerating .surface to the shell script's curated shape (the literal ask) would have:

permanently reddened TestSurfaceSnapshot (the two generators can't both own the file),
churned ~9k lines,
forced ~9 false-drift baselines into .surface-skill-drift for globals (--jq, --agent, --profile) that do work on those subcommands — curation drops true information, and
enshrined an agent-help bug (below) as canonical surface.

What this PR changes instead (4 files, zero `.surface` churn)

1. Fix the agent-help hidden-flag leak (internal/cli/root.go)
The "2 additions" #539 noticed (FLAG basecamp recordings [list] --assignee) are a leak, not new surface. --assignee is MarkHidden with usage "Not supported — use reports assigned instead". emitAgentHelp surfaced it anyway because pflag's VisitAll visits hidden flags. Skip hidden flags at all three emission sites so agent-help matches text --help and the tree walker. Legitimate visible--assignee/--assignees flags on cards/todos are unaffected. Regression test in help_test.go.

2. Make check-surface a real gate (Makefile)
It regenerated to /tmp and echoed a count — caught nothing but a totally broken binary. Repoint it at TestSurfaceSnapshot (the authority) so it fails on drift and points at a new make update-surface regeneration target. check-surface-compat / the cli-surface CI job are untouched (they diff generated-vs-generated across versions and never read the committed file).

3. Idempotent regeneration (internal/commands/surface_test.go)
-update-surface wrote .surface without a trailing newline (stripping the POSIX one, creating churn). Add one so make update-surface is idempotent.

Verification

make check-surface passes, and bites when .surface drifts (pointing at make update-surface).
make update-surface is idempotent (16,586 lines, no diff).
basecamp recordings --help --agent no longer emits --assignee; the shell script's output drops the 2 leak lines while keeping the 8 legit visible ones.
TestSurfaceSnapshot, check-skill-drift, check-smoke-coverage green; bin/ci exits 0.

Follow-up note for #539

The shell script and .surface still legitimately differ on the curated-vs-full-globals axis, but no consumer cross-checks them, so it's benign. If we later want a single source of truth, the options are: (a) keep the Go generator as authority (status quo after this PR), (b) migrate the snapshot + compat gate onto the shell script and retire TestSurfaceSnapshot/.surface-breaking, or (c) reconcile the two generators in github.com/basecamp/cli. This PR takes (a) as the smallest correct step.

Summary by cubic

Fixes the hidden flag leak in agent help and turns check-surface into a real drift gate bound to TestSurfaceSnapshot. Also hardens drift detection to catch acknowledged-removal-only staleness and keeps update-surface idempotent, addressing #539 without churn.

Bug Fixes
- Agent help now skips hidden flags across all emission paths; strengthened TestAgentHelpOmitsHiddenFlags to assert success and verify both flags and inherited_flags.
Refactors
- check-surface runs TestSurfaceSnapshot and now fails when .surface byte-differs with no additions or unacknowledged removals (baselineNeedsRegen); make help updated.
- update-surface only rewrites .surface when the snapshot differs and no unacknowledged removals remain (shouldWriteBaseline), still writes on removal-only updates, and appends a trailing newline; added TestShouldWriteBaseline and TestBaselineNeedsRegen.

^{Written for commit d7e7d00. Summary will update on new commits.}

The committed .surface snapshot is generated and enforced by the Go surface package (surface.SnapshotString) via TestSurfaceSnapshot, not by scripts/check-cli-surface.sh. The two generators diverge: the Go one walks the command tree (full inherited globals, excludes hidden flags, no --version leak), while the shell script walks --help --agent (curated salient globals post-#499, and — the bug here — emits hidden flags). Rather than churn .surface to the script's shape (which would break TestSurfaceSnapshot and enshrine the leak), fix the underlying issues: - emitAgentHelp leaked hidden flags because pflag's VisitAll visits them, so `basecamp recordings --help --agent` surfaced the MarkHidden'd, "Not supported" --assignee flag. Skip hidden flags at all three emission sites to match text --help and the tree walker. Add a regression test. - check-surface previously regenerated to /tmp and echoed a line count, catching nothing. Repoint it at TestSurfaceSnapshot (the authority) so it actually fails on drift, and add an update-surface target for regeneration. check-surface-compat / the cli-surface CI job are unchanged (they diff generated-vs-generated across versions). - TestSurfaceSnapshot's -update-surface wrote .surface without a trailing newline; add one so update-surface is idempotent and POSIX-clean. Closes#539.

Copilot

Pull request overview

This PR addresses #539 by fixing an agent-help bug (hidden flags leaking into --help --agent JSON output) and by turning make check-surface into a real drift gate tied to the authoritative TestSurfaceSnapshot, while keeping the committed .surface snapshot owned by the Go generator (no churn).

Changes:

Filter hidden flags out of agent-help output so --help --agent matches text help behavior.
Rework make check-surface to run TestSurfaceSnapshot and add an update-surface target for regeneration guidance.
Make .surface regeneration idempotent by ensuring it is written with a trailing newline.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
Makefile	Makes `check-surface` enforce `.surface` drift via `TestSurfaceSnapshot` and adds an `update-surface` target.
internal/commands/surface_test.go	Ensures regenerated `.surface` includes a trailing newline for idempotence.
internal/cli/root.go	Filters hidden flags out of agent-help JSON emission across local/parent-scoped/inherited flag sources.
internal/cli/help_test.go	Adds a regression test asserting hidden flags (e.g. recordings `--assignee`) do not appear in agent-help output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:21a3d68ee9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cubic-dev-ai

All reported issues were addressed across 4 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

TestSurfaceSnapshot's -update-surface only wrote .surface when there were additions, so a removal-only change (removed lines acknowledged in .surface-breaking) left the removed CMD/FLAG lines in .surface — and consumers that read it as the current surface (check-skill-drift, check-smoke-coverage) would keep validating removed commands. Write whenever there are no unacknowledged removals instead.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Refine the -update-surface write condition into a testable predicate, shouldWriteBaseline: write when update mode is on, there are no unacknowledged removals, and the baseline bytes differ from the freshly generated surface. The byte-difference guard keeps removal-only updates (acknowledged in .surface-breaking) writing while making an unchanged surface a true no-op. Add a TestShouldWriteBaseline regression test covering removal-only, addition, no-change, unacknowledged-removal, and non-update cases. Also refresh 'make help': describe check-surface as the drift gate and list the new update-surface target.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:c209032cd5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- check-surface now fails when the committed .surface differs from the generated surface even if the only delta is an acknowledged removal that was never regenerated (baselineNeedsRegen). Previously such a baseline passed the gate while still feeding stale removed lines to check-skill-drift and check-smoke-coverage. - TestAgentHelpOmitsHiddenFlags now asserts Execute() succeeds and checks both flags and inherited_flags, matching the guarantee that no hidden flag appears anywhere in agent help. - check-surface's failure hint is now conditional (TestSurfaceSnapshot can fail for reasons other than drift). Add TestBaselineNeedsRegen covering the acknowledged-removal, no-change, addition, and unacknowledged-removal cases.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

…e-codex-plugin * origin/main: Bump SDK to d15f023f and collapse todos update onto TodosService.Edit (basecamp#543) Fix agent-help hidden-flag leak; make check-surface a real drift gate (basecamp#542) Fix todos update silently clearing completion subscribers (basecamp#540) Fix TestHardenConfigDir on macOS: resolve TMPDIR symlinks in test dirs (basecamp#536) Bump golang.org/x/net to v0.56.0 for CVE-2026-46600 (basecamp#537)

CopilotAI review requested due to automatic review settings July 18, 2026 06:35

github-actionsBot added commands CLI command implementations tests Tests (unit and e2e) bug Something isn't working labels Jul 18, 2026

Copilotstarted reviewing on behalf of jeremy July 18, 2026 06:36 View session

CopilotAI reviewed Jul 18, 2026
View reviewed changes

Comment threadMakefile

chatgpt-codex-connectorBot reviewed Jul 18, 2026
View reviewed changes

Comment threadMakefile

cubic-dev-aiBot reviewed Jul 18, 2026
View reviewed changes

Comment threadMakefile

CopilotAI review requested due to automatic review settings July 18, 2026 06:50

Copilotstarted reviewing on behalf of jeremy July 18, 2026 06:50 View session

CopilotAI reviewed Jul 18, 2026
View reviewed changes

Comment threadinternal/commands/surface_test.go
Comment threadinternal/cli/help_test.go

CopilotAI review requested due to automatic review settings July 18, 2026 07:01

Copilotstarted reviewing on behalf of jeremy July 18, 2026 07:01 View session

CopilotAI reviewed Jul 18, 2026
View reviewed changes

Comment threadMakefile
Comment threadinternal/cli/help_test.go

chatgpt-codex-connectorBot reviewed Jul 18, 2026
View reviewed changes

Comment threadMakefile

CopilotAI review requested due to automatic review settings July 18, 2026 20:35

Copilotstarted reviewing on behalf of jeremy July 18, 2026 20:36 View session

CopilotAI reviewed Jul 18, 2026
View reviewed changes

jeremy merged commit b7c8e91 into mainJul 19, 2026
27 checks passed

jeremy deleted the stale-surface branch July 19, 2026 18:06

Navigation Menu

Uh oh!

Fix agent-help hidden-flag leak; make check-surface a real drift gate - #542

Fix agent-help hidden-flag leak; make check-surface a real drift gate#542
jeremy merged 4 commits into
mainfrom
stale-surface

jeremy commented Jul 18, 2026•
edited by cubic-dev-ai Bot
Loading

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Uh oh!

Uh oh!

cubic-dev-aiBot left a comment •
edited
Loading

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Uh oh!

Uh oh!

CopilotAI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

jeremy commented Jul 18, 2026• edited by cubic-dev-ai BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

What's actually going on

What this PR changes instead (4 files, zero .surface churn)

Verification

Follow-up note for #539

Summary by cubic

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

cubic-dev-aiBot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jeremy commented Jul 18, 2026•
edited by cubic-dev-ai Bot
Loading

What this PR changes instead (4 files, zero `.surface` churn)

cubic-dev-aiBot left a comment •
edited
Loading