Skip to content

Replace negative-priming prohibition with positive framing in consequences field - #471

Closed
neoneye wants to merge 1 commit into
mainfrom
fix/identify-remove-negative-priming
Closed

Replace negative-priming prohibition with positive framing in consequences field#471
neoneye wants to merge 1 commit into
mainfrom
fix/identify-remove-negative-priming

Conversation

@neoneye

Copy link
Copy Markdown
Member

Summary

Replace "Do NOT include 'Controls ... vs.', 'Weakness:'" with "Focus on cause-effect relationships and factual outcomes; save critical assessments for the review_lever field." in the consequences field description of both Lever and LeverCleaned Pydantic models.

The negative prohibition names exact banned phrases that small models copy as templates — the same anti-pattern that caused regressions in the enrich step (PRs #458, #460). Positive framing guides models toward desired behavior without priming the unwanted pattern.

Test plan

  • Run self_improve experiment for identify_potential_levers step
  • Verify llama3.1 consequences field no longer contains "Controls ... vs." or "Weakness:" patterns
  • Verify no regression in consequences quality for other models
  • Check assessment verdict before merging

🤖 Generated with Claude Code

…ences field
Replace "Do NOT include 'Controls ... vs.', 'Weakness:'" with "Focus
on cause-effect relationships and factual outcomes; save critical
assessments for the review_lever field." in both Lever and LeverCleaned
Pydantic models.
The negative prohibition names exact banned phrases that small models
(llama3.1) copy as templates — the same anti-pattern that caused
regressions in the enrich step (PRs #458, #460). Positive framing
guides the model toward the desired behavior without priming the
unwanted pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@neoneye

Copy link
Copy Markdown
MemberAuthor

Self-improve iteration — analysis 69

Verdict: YES

  • Positive framing correctly replaces negative prohibition in both Lever and LeverCleaned models
  • "Controls X vs Y" / "Weakness:" patterns not observed in any after-batch outputs
  • No content regressions — consequences field produces clean cause-effect prose
  • Success rate: 97.8% (excl. gpt-oss-20b infrastructure timeout, not PR-caused)
  • OPTIMIZE_INSTRUCTIONS alignment improved (lines 80-83 no longer violated)

@neoneye

Copy link
Copy Markdown
MemberAuthor

Closing — gpt-oss-20b timed out on all 5 plans (infrastructure issue needs investigation before merging).

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@neoneye