Skip to content

Adaptive batch size, guarded retry, and OpenRouter config fixes for enrich step - #456

Merged
neoneye merged 1 commit into
mainfrom
fix/enrich-adaptive-batch-and-retry-guards-v4
Mar 30, 2026
Merged

Adaptive batch size, guarded retry, and OpenRouter config fixes for enrich step#456
neoneye merged 1 commit into
mainfrom
fix/enrich-adaptive-batch-and-retry-guards-v4

Conversation

@neoneye

Copy link
Copy Markdown
Member

Summary

  • OpenRouter config fixes: Add explicit context_window for all 4 OpenRouter models (131072/1048576/128000/40960) — LlamaIndex defaults to 3900 for all OpenRouter models without this
  • gpt-oss-20b max_tokens: 8192→65536 (balanced: 65536 input + 65536 output within 131K context)
  • Adaptive batch size: batch_size=2 when context_window < 3000 (threshold below OpenRouter's 3900 fallback to avoid false positives)
  • Guarded retry: Split once (depth=1) within 300s budget, then skip
  • Error tracking: Persist errors in raw JSON output for offline diagnosis
  • Accurate batch counting: batches_succeeded instead of hardcoded 1
  • OPTIMIZE_INSTRUCTIONS: Document 4 new known problems including OpenRouter metadata fallback

Supersedes #453, #454, #455 which each had config or threshold bugs.

Test plan

  • Verify all 7 models use batch_size=5 (no model has context_window < 3000)
  • Verify gpt-oss-20b achieves 5/5 plan success with valid output
  • Verify no BadRequestError in error arrays
  • Verify qwen3-30b, gpt-4o-mini, gemini-flash use calls_succeeded=2 (ceil(7/5)=2, not 3-4)
  • Check assessment verdict before merging

🤖 Generated with Claude Code

…nrich step
Fix all OpenRouter models in baseline.json: add explicit context_window
(131072/1048576/128000/40960) so LlamaIndex sees real values instead of
its 3900 fallback. Bump gpt-oss-20b max_tokens 8192→65536 (balanced
headroom: 65536 input + 65536 output within 131K context).
Add adaptive batch sizing: batch_size=2 when context_window < 3000
(threshold below OpenRouter's 3900 fallback to avoid false positives).
Add guarded retry: split once (MAX_RETRY_DEPTH=1) within 300s budget,
then skip. Persist errors in raw JSON output. Report actual
batches_succeeded instead of hardcoded 1.
Document consequence-echoing, UUID format inconsistency, max_tokens
overflow, and OpenRouter metadata fallback in OPTIMIZE_INSTRUCTIONS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@neoneye

Copy link
Copy Markdown
MemberAuthor

Self-improve iteration — analysis 59

Verdict: CONDITIONAL

Wins (all 6 targeted issues resolved):

  • gpt-oss-20b: 0/5 → 4/5 plans (+11.4pp overall success rate, largest single-iteration gain)
  • OpenRouter context_window: all 4 models now report real values (131K/1M/128K/41K)
  • Adaptive batch size: SMALL_CONTEXT_THRESHOLD=3000 correctly avoids false positives on OpenRouter's 3900 fallback
  • Guarded retry: implemented, not triggered (all batches succeeded first attempt)
  • Error tracking: surfaced pre-existing llama3.1 phantom lever ID bug (3 levers silently unenriched)
  • Accurate batch counting: working correctly

Remaining issues (not regressions):

  • gpt-oss-20b parasomnia timeout at 600s (model throughput limitation, not config issue — gta_game took 243s vs 21-47s for other models)
  • llama3.1 phantom lever IDs: 3/35 levers unenriched due to truncated UUIDs (pre-existing, now visible thanks to error tracking)

No content regressions introduced.

@neoneye
neoneye merged commit 39f6316 into mainMar 30, 2026
3 checks passed
@neoneye
neoneye deleted the fix/enrich-adaptive-batch-and-retry-guards-v4 branch April 1, 2026 23:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@neoneye