reflect: prove pipelines by their real output, not a substitute - #297
Open
EdbertChan wants to merge 1 commit into
Conversation
Four fixes in one night each passed a cheap proof and then failed one layer down on the real path: a unit test, a config read-back, four static workflow assertions, and a dry run. Every fix was correct; none of them made the feature work. Adds three evidence rules: a pipeline change is proven only by a real run that emits its real output, a dry run never proves a write path, and an identity/binary/database change must be verified as the service resolves it rather than as the operator's shell does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVGJAQuwdDArq4oGK2wrSE Change-Id: Ib661b78e918424539f46bf8d421cedabfba92c86
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_35c2c5d7-b39f-4a84-a0a7-7602380ffa37) |
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four fixes on the night of Sep 8 each passed a cheap proof and then failed one layer down when the real path ran. Every fix was correct. None of them made the feature work.
The cheap proofs were a unit test, a config read-back, four static workflow assertions, and a dry run. Each stands in for the real path, and each defect lived exactly at the seam the stand-in skips.
This adds three evidence rules to the always-loaded corpus.
The four instances
GitHub Actions is not permitted to create or approve pull requestsfiled 2submit failed,filed 0Rules added
Test plan
corpus/CLAUDE.learned.mdrenders and the rules sit under the existing Evidence heading