Add a frozen concise-review adaptation experiment

[Copilot]

Adaptive review benefit remains unproven: issue drafts may contain excessive context, but runtime expansion must stay paused until a concise candidate beats a baseline without losing required facts or actions.

• Frozen experiment

  • Adds two development and six held-out public issue-review cases.
  • Preregisters comprehension, editing, preservation, human-preference, and judge-agreement thresholds.
  • Records the feedback signal as user-directed without claiming recurrence evidence.

• Reproducible candidate

  • Compiles isolated baseline and concise-review agents through agent-compiler.
  • Pins corpus, experiment, AgentImage, rendered-agent, prompt, and commit provenance.
  • Keeps the candidate read-only and outside the marketplace runtime.

• Evidence contract

  • Generates salted, randomly ordered blind-review sheets containing the frozen rubric and requirements.
  • Retains raw paired outputs and total token/cost usage.
  • Requires a separately authorized, input-bound frontier judge from another model family.
  • Rejects incomplete, duplicated, contaminated, truncated, unqualified, or overspent evidence.

• Decision boundary

  • Reports improvement, regression, or inconclusive.
  • Never installs automatically; improvement only becomes eligible for scoped human approval.
  • Defines monitoring and baseline restoration before any later runtime expansion.
  • Records the current result as inconclusive because no qualified actor or authorized judge run exists.

evals/paid/adaptive-review/evaluate.py blind \
  --run runs/<id>/run.json \
  --out runs/<id>/human-labels.json

evals/paid/adaptive-review/evaluate.py evaluate \
  --run runs/<id>/run.json \
  --human runs/<id>/human-labels.json \
  --judge runs/<id>/judge-labels.json \
  --out runs/<id>/report.json

• Fixes Adaptive behavior v1: learn from feedback and prove one useful improvement #117