Skip to content

reflect: prove pipelines by their real output, not a substitute - #297

Open
EdbertChan wants to merge 1 commit into
mainfrom
stack/EdbertChan/reflect/pipeline-real-path-proof/prove-pipelines-their-real-output-substitute--b661b78e
Open

reflect: prove pipelines by their real output, not a substitute#297
EdbertChan wants to merge 1 commit into
mainfrom
stack/EdbertChan/reflect/pipeline-real-path-proof/prove-pipelines-their-real-output-substitute--b661b78e

Conversation

@EdbertChan

@EdbertChanEdbertChan commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Summary

Four fixes on the night of Sep 8 each passed a cheap proof and then failed one layer down when the real path ran. Every fix was correct. None of them made the feature work.

The cheap proofs were a unit test, a config read-back, four static workflow assertions, and a dry run. Each stands in for the real path, and each defect lived exactly at the seam the stand-in skips.

This adds three evidence rules to the always-loaded corpus.

The four instances

FixCheap proof that passedWhat the real path did
Repair-plan header (Invoker #12007)unit testdied at dispatch on an agent/model mismatch
Fleet model switchconfig read-backkilled the owner at boot on a third key that overrides the edited one
Nightly release (Invoker #12023)four static workflow assertionsGitHub Actions is not permitted to create or approve pull requests
Session miner (Invoker #12027)dry run reporting filed 2submit failed, filed 0

Rules added

  1. A pipeline or worker change is proven only by one uninterrupted real run at production settings that emits the pipeline's real output: a workflow id, a PR number, a tag, a merged commit.
  2. A dry run proves the decision, never the effect. Never accept one as proof of a write path.
  3. When a fix changes which identity, binary, or database a process uses, verify what the service resolves, not what your shell resolves.

Test plan

  • corpus/CLAUDE.learned.md renders and the rules sit under the existing Evidence heading
  • Backtest on the next pipeline change: its acceptance line should be a real output id

Four fixes in one night each passed a cheap proof and then failed one
layer down on the real path: a unit test, a config read-back, four
static workflow assertions, and a dry run. Every fix was correct; none
of them made the feature work.
Adds three evidence rules: a pipeline change is proven only by a real
run that emits its real output, a dry run never proves a write path,
and an identity/binary/database change must be verified as the service
resolves it rather than as the operator's shell does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVGJAQuwdDArq4oGK2wrSE
Change-Id: Ib661b78e918424539f46bf8d421cedabfba92c86
@cursor

cursorBot commented Sep 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_35c2c5d7-b39f-4a84-a0a7-7602380ffa37)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@EdbertChan