fix(prediction): isolate settlement verdicts

[proerror77]

Change contract

Governed Polymarket settlement keep/discard feedback uses one event-root UP probability sample per decision and cannot be changed by token fillability, depth, slippage, capacity, or execution PnL evidence.

Out of scope

Full-update-tick collection; UP/DOWN repricing, fill, slippage, or markout heads; 15m and 1h missions; collector/ECS changes; renaming the legacy mission target; changing the existing execution-sensitive promotion report.

Dependencies and merge order

None.

Focused validation

• Targeted counterexample builds identical event-root probability/label/cohort inputs with healthy versus failed execution evidence and requires byte-identical governed feedback.
• Targeted cardinality test requires one UP event-root settlement sample per source decision rather than duplicated UP/DOWN labels.
• Targeted baseline test requires incremental Brier, log loss, and calibration improvement.
• Independent code and scope reviews: approved with no hard findings.
• rustfmt --check, jq parsing, and git diff --check: passed.
• Linux compilation and tests: GitHub Actions only, pending on this PR.
• Policy identity independently reproduced as sha256:f1768c939aecd9e49ebe6c74c46d884d139ee51539d141cea5e6e87d787d6717 and pinned by both mission templates.

Rollout and rollback

Research evaluator and immutable mission-policy identity only; no collector, ECS, live execution, or deployment mutation. Roll back by reverting this PR, which restores the prior execution-conditioned feedback semantics and prior policy identity.

Scope exception

None.

Summary by CodeRabbit

• New Features

  • Added settlement-verdict walk-forward reporting with out-of-sample probability quality and calibration metrics.
  • Prediction research feedback now incorporates verdict pass rates, baseline comparisons, and additional candidate evidence.

• Improvements

  • Evaluation results now use event-level decision samples for more consistent settlement analysis.
  • Refreshed research policy snapshots for BTC and SOL five-minute market configurations.

• Bug Fixes

  • Ensured prediction feedback remains unchanged when unrelated execution evidence changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7e816a86-f797-437c-bb15-c5a14054a89d

📥 Commits

Reviewing files that changed from the base of the PR and between 7e6c455 and 33b1d58.

📒 Files selected for processing (5)

• rust_hft/prediction-markets/config/research_missions/polymarket-btc-5m.example.json
• rust_hft/prediction-markets/config/research_missions/polymarket-sol-5m.example.json
• rust_hft/prediction-markets/crates/ploy-research/src/bin/monday-prediction-evaluator.rs
• rust_hft/prediction-markets/crates/ploy-research/src/factors_v2.rs
• rust_hft/prediction-markets/crates/ploy-research/src/lib.rs

---

📝 Walkthrough

Walkthrough

The PR adds a settlement-verdict walk-forward pipeline using out-of-sample probability-quality metrics, updates prediction feedback to use calibration-based gates, wires the evaluator to generate verdict reports, updates related tests, and changes two mission snapshot identifiers.

Changes

Settlement verdict research

Layer / File(s)	Summary
Verdict report contracts and fitting inputs rust_hft/prediction-markets/crates/ploy-research/src/factors_v2.rs, rust_hft/prediction-markets/crates/ploy-research/src/lib.rs	Adds settlement-verdict window, aggregate, and report types; re-exports them; and changes EventVolSurface::fit to accept (row, win) pairs.
Verdict walk-forward computation rust_hft/prediction-markets/crates/ploy-research/src/factors_v2.rs	Adds event-root UP sampling, probability-surface fitting, OOS Brier/log-loss/ECE calculations, pass gates, and aggregate report generation.
Calibration-based feedback decisions rust_hft/prediction-markets/crates/ploy-research/src/factors_v2.rs	Updates prediction feedback to consume verdict reports, compare baseline calibration, emit OOS metrics, and test execution-evidence isolation and incremental calibration behavior.
Evaluator integration and mission snapshots rust_hft/prediction-markets/crates/ploy-research/src/bin/monday-prediction-evaluator.rs, rust_hft/prediction-markets/config/research_missions/*.example.json, rust_hft/prediction-markets/crates/ploy-research/src/factors_v2.rs	Generates and passes the settlement-verdict report during evaluation, updates related tests, and changes BTC and SOL policy snapshot identifiers.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Evaluator as monday-prediction-evaluator
  participant Report as walk_forward_settlement_verdict_report_with_prior
  participant Feedback as build_prediction_research_feedback
  Evaluator->>Report: generate settlement verdict walk-forward report
  Report-->>Evaluator: return OOS calibration report
  Evaluator->>Feedback: pass verdict report
  Feedback-->>Evaluator: return calibration-based prediction feedback

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: isolating settlement verdict behavior.
Description check	✅ Passed	The description follows the required template and includes all required sections with relevant details.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/polymarket-settlement-verdict-isolation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}