Skip to content

feat(research): allow baseline-only prediction missions - #156

Merged
proerror77 merged 1 commit into
mainfrom
codex/prediction-baseline-only-budget
Jul 19, 2026
Merged

feat(research): allow baseline-only prediction missions#156
proerror77 merged 1 commit into
mainfrom
codex/prediction-baseline-only-budget

Conversation

@proerror77

@proerror77 proerror77 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Change contract

Allow an exact max_candidates=0, max_llm_calls=0, positive-time prediction mission to execute its governed baseline exactly once and terminate before any proposal/LLM call.

Out of scope

Evaluator/report artifact changes, cohort windowing, candidate replay, live trading, and any relaxation of governed positive search budgets.

Dependencies and merge order

None. This focused PR is based on current main at c4870199 and may merge independently.

Focused validation

  • Added a validation counterexample proving partial-zero (0/N, N/0) budgets and 0/0/0 remain fail-closed.
  • Added a baseline-only loop test proving one evaluator call, zero proposal calls, zero candidates, no LLM-call directory, valid baseline evidence, and deterministic budget exhaustion.
  • Independent read-only code review: approved with no findings.
  • cargo fmt --package ploy-research and git diff --check: pass.
  • Local build/tests intentionally not run; Linux CI is the execution authority for this repository.
  • Scope is 3 changed files and 129 diff lines (including two required policy-hash pins), below the repository review thresholds.

Rollout and rollback

No live execution impact. Baseline-only behavior is opt-in through an exact zero/zero budget; rollback is reverting this commit to restore the previous positive-only validation.

Scope exception

None.

Summary by CodeRabbit

  • New Features

    • Added support for baseline-only prediction missions with explicit budget controls.
    • Baseline-only missions now run once using the baseline evaluator without proposal generation or language-model calls.
  • Bug Fixes

    • Improved validation for mission budgets, including positive time limits and consistent candidate and model-call limits.
    • Updated BTC and SOL mission configurations to use the latest search policy snapshots.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The PR updates BTC and SOL Polymarket mission policy snapshot identifiers and adds baseline-only search-budget validation. Tests verify invalid budget combinations and confirm baseline-only runs execute once without proposal or LLM-call activity.

Changes

Baseline-only mission execution

Layer / File(s) Summary
Mission policy snapshot updates
rust_hft/prediction-markets/config/research_missions/*.example.json
BTC and SOL five-minute mission templates reference new search_policy_snapshot_id values.
Baseline budget validation and execution
rust_hft/prediction-markets/crates/ploy-research/src/prediction_loop.rs
Validation requires positive max_seconds and paired zero values for baseline-only budgets; tests verify validation errors and deterministic terminal execution without proposal or LLM calls.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant run_or_resume
  participant validate_prediction_mission
  participant baseline_evaluator
  participant proposal_client
  participant loop_state
  run_or_resume->>validate_prediction_mission: validate baseline-only budget
  validate_prediction_mission-->>run_or_resume: accept valid budget
  run_or_resume->>baseline_evaluator: execute baseline once
  run_or_resume-->>proposal_client: no proposal call
  run_or_resume->>loop_state: persist BudgetExhausted terminal state
Loading

Possibly related PRs

  • proerror77/monday#76: Pins a reviewed prediction-policy graph identity related to the updated snapshot identifiers.
  • proerror77/monday#139: Changes policy-source inputs used by prediction policy snapshot identities.
  • proerror77/monday#145: Updates the same Polymarket five-minute mission template snapshot identifiers.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: enabling baseline-only prediction missions.
Description check ✅ Passed The description matches the template and includes all required sections with concrete details.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/prediction-baseline-only-budget

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proerror77
proerror77 force-pushed the codex/prediction-baseline-only-budget branch from 40f68f1 to fd4804d Compare July 19, 2026 12:07

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 40f68f1bd4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if mission.search_budget.max_candidates == 0
|| mission.search_budget.max_llm_calls == 0
|| mission.search_budget.max_seconds == 0
if mission.search_budget.max_seconds == 0 {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Refresh the checked-in policy snapshot IDs

Because prediction_policy_sources() hashes this entire file (lines 293–294), this edit changes the current policy ID from the templates' sha256:5d1512… to sha256:cdacb8…, but neither checked-in BTC/SOL mission JSON was updated. Consequently, the existing test at lines 4015–4018 fails, and operators using either template will be rejected by validate_prediction_mission for a stale search_policy_snapshot_id; update both template IDs alongside this hashed policy change.

Useful? React with 👍 / 👎.

@proerror77
proerror77 merged commit cac45b9 into main Jul 19, 2026
28 checks passed
@proerror77
proerror77 deleted the codex/prediction-baseline-only-budget branch July 19, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant