Uh oh!
There was an error while loading. Please reload this page.
Comprehensive prompt refinement: verbatim numbers, template-lock fix, tighter targets - #479
Closed
neoneye wants to merge 1 commit into
Closed
Comprehensive prompt refinement: verbatim numbers, template-lock fix, tighter targets#479neoneye wants to merge 1 commit into
neoneye wants to merge 1 commit into
Conversation
… tighter targets Bundles all proven improvements from analyses 69-73 into one PR: 1. Verbatim-numbers constraint on BOTH consequences AND options: "Use numbers only when the project context provides them directly — do not calculate, derive, or estimate figures." Closes the arithmetic-derivation loophole (HK$470M→HK$141M). 2. Fix review_lever template lock (#1 issue from analysis 40): Replace "state the specific gap the three options leave unaddressed" with "identify the key risk or constraint that the proposed options collectively do not resolve." Removes "the three options" as grammatical anchor that caused 70-94% haiku template lock. 3. Positive framing: replace "Do NOT include 'Controls ... vs.'" with "Save critical assessments for the review_lever field." 4. Consistent targets everywhere: consequences 2-3 sentences, options one sentence, review_lever one sentence (20-40 words). All field descriptions and system prompt sections aligned. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
neoneye
commented
Apr 1, 2026
MemberAuthor
Self-improve iteration — analysis 74Verdict: CONDITIONAL Win:
Regressions:
The fundamental problem: replacing one copyable phrase with another. Any sentence-structural phrase in the field description becomes a template for small models. The assessment cites OPTIMIZE_INSTRUCTIONS lines 86-92 — the PR violated its own documented anti-pattern. |
neoneye
commented
Apr 1, 2026
MemberAuthor
Closing — replaced one template lock with another. See iteration_insights.md in analysis 74. |
neoneye added a commit
that referenced
this pull request
Apr 1, 2026
Same as PR #479 (verbatim numbers, positive framing, consistent targets) but with one tweak: strip review_lever field description to bare minimum "Critical review of this lever (one sentence, 20-40 words)" with no structural phrase that models can copy as a template. PR #479's "the proposed options collectively do not resolve" caused qwen3-30b to lock (0→7/20) and worsened gpt-oss-20b (10/17→15/17). This version removes ALL copyable sentence structures from review_lever and lets the section 4 examples teach the desired style instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 tasks
neoneye added a commit
that referenced
this pull request
Apr 1, 2026
Document proven failure modes and fixes from 10 self_improve iterations: - Field-description template lock: expanded with evidence from PR #484 (the fix) and PR #479 (replacement phrase also locks). Strip to content-type + word count, let examples teach. - Negative prohibitions activate banned patterns: confirmed across PRs #458, #460, #475. Use positive framing instead. - Consequence parroting in later calls: llama3.1 copies consequences into review_lever. Anti-parrot needed in subsequent-call prompt. - Field descriptions vs system prompt consistency: haiku follows system prompt, text-completion models follow field descriptions. - Stripping field descriptions too far: weak models need "one sentence" and section pointer as structural anchors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bundles all proven improvements from analyses 69-73 into one definitive PR:
Supersedes #471, #473, #475, #477, #478.
Test plan
🤖 Generated with Claude Code