Skip to content

fix: remove 'core tension' template lock from field description - #358

Merged
neoneye merged 1 commit into
mainfrom
fix/remove-core-tension-template-lock
Mar 19, 2026
Merged

fix: remove 'core tension' template lock from field description#358
neoneye merged 1 commit into
mainfrom
fix/remove-core-tension-template-lock

Conversation

@neoneye

Copy link
Copy Markdown
Member

Summary

The phrase "name the core tension" in the review_lever field description (line 111) and system prompt section 4 (line 228) caused ~100% "The tension is between X and Y" opener lock across all plans for llama3.1 and haiku (confirmed in iter 39, analysis 39).

  • Field description: "name the core tension, then identify a weakness" → "identify the primary trade-off this lever introduces, then state the specific gap"
  • Section 4 header: same rewrite
  • Section 6: "under 2 sentences (aim for 20-40 words)" → "one sentence (20-40 words)" — removes ambiguity
  • OPTIMIZE_INSTRUCTIONS: document field-description template lock as a known failure mode

The three section-4 examples already use domain-specific openers (not "The tension is..."), so removing the structural cue from the instruction text aligns instruction with examples.

Test plan

  • Run self_improve iteration (7 models × 5 plans)
  • Verify "The tension is between X and Y" opener drops from ~100% to <40% for haiku
  • Verify llama3.1 reviews use diverse openers instead of uniform "tension" pattern
  • Verify success rate stays at 100%
  • Verify no regression in silo/hong_kong template lock (should remain at 0%)

🤖 Generated with Claude Code

…rompt
The phrase "name the core tension" in the review_lever field description
and section 4 header caused ~100% "The tension is between X and Y"
opener lock across all plans for llama3.1 and haiku.
Replace with content-goal language: "identify the primary trade-off...
then state the specific gap" — describes what to produce, not what
sentence structure to use.
Also: tighten section 6 review cap from "under 2 sentences" to
"one sentence (20-40 words)" and document field-description template
lock in OPTIMIZE_INSTRUCTIONS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@neoneye

Copy link
Copy Markdown
MemberAuthor

Self-Improve Iteration 40 — Verdict: YES ✅

"Tension" template lock eliminated.

MetricBefore (iter 39)After (iter 40)
"Tension" lock (haiku hong_kong)75%0%
"Tension" lock (llama3.1)~100%~0%
Haiku call success rate73.3%86.7% (+13.4pp)
Overall call success rate95.2%97.1% (+1.9pp)

Secondary lock emerged

  • "None/All three options" at ~85% for haiku — caused by "the three options leave unaddressed" in new field description
  • Same template-lock migration mechanism, milder form — target for next iteration

Minor cleanup needed

  • LeverCleaned.review field description (line 212) still says "core tension" — not sent to LLM, cosmetic only

Data: PlanExe-prompt-lab/analysis/40_identify_potential_levers/

@neoneye
neoneye merged commit d035717 into mainMar 19, 2026
3 checks passed
@neoneye
neoneye deleted the fix/remove-core-tension-template-lock branch March 19, 2026 17:56
neoneye added a commit that referenced this pull request Apr 1, 2026
Remove "identify the primary trade-off this lever introduces, then
state the specific gap the three options leave unaddressed" from two
locations: the Pydantic field description and the system prompt
section 4 preamble. Replace with minimal word-count guidance.
This phrase caused 85% template lock in haiku reviews ("All three
options X, but none address Y"). Proven fix from analysis 76 (#482).
All other field descriptions, system prompt sections, and prohibitions
unchanged from baseline (PR #358).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@neoneye