Skip to content

Bind Polymarket evidence to trade completion proofs - #152

Merged
proerror77 merged 8 commits into
mainfrom
codex/polymarket-trade-tape-replay
Jul 19, 2026
Merged

Bind Polymarket evidence to trade completion proofs#152
proerror77 merged 8 commits into
mainfrom
codex/polymarket-trade-tape-replay

Conversation

@proerror77

@proerror77 proerror77 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Change contract:

  • Bind every selected Polymarket 5m evidence trade end-to-end to an event-local collector completion proof aggregated across the validated consecutive reference segment set; reject missing, duplicate, or mismatched identities while tolerating unrelated 5m/15m and other-symbol proofs in the same source set.

Out of scope:

Dependency or merge order:

Focused validation:

  • cargo test --manifest-path rust_hft/Cargo.toml -p hft-collector --lib -> 174 passed, 2 ignored.
  • cargo test --manifest-path rust_hft/prediction-markets/Cargo.toml -p ploy-market-data -> 39 passed.
  • cargo test --manifest-path rust_hft/prediction-markets/Cargo.toml -p ploy-research -> 248 library tests and 2 binary tests passed.
  • cargo clippy --manifest-path rust_hft/Cargo.toml -p hft-collector --all-targets -- -D warnings -> passed.
  • cargo clippy --manifest-path rust_hft/prediction-markets/Cargo.toml -p ploy-market-data --all-targets --no-deps -- -D warnings -> passed.
  • git diff --check -> passed.
  • Targeted counterexamples cover cross-hour proof movement, duplicate proof identities, missing proof/marker counts, exact count/digest mismatch, unselected trade rows, mixed 5m/15m and extra-symbol proofs, and legacy unqualified evidence schema rejection.
  • Two fixed-head reviewers at base 9270aef8 and head 78f62f45 -> no findings.
  • Scope: 9 files, 745 changed lines; below the 25-file/750-line split threshold.

Rollout/rollback impact:

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@proerror77, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 873d267d-3482-42ed-b980-0dd680ad419e

📥 Commits

Reviewing files that changed from the base of the PR and between 0ba046b and b9ec3d8.

📒 Files selected for processing (7)
  • rust_hft/prediction-markets/config/research_missions/polymarket-btc-5m.example.json
  • rust_hft/prediction-markets/config/research_missions/polymarket-sol-5m.example.json
  • rust_hft/prediction-markets/crates/ploy-market-data/src/polymarket_evidence/artifact.rs
  • rust_hft/prediction-markets/crates/ploy-research/src/polymarket_evidence_projection.rs
  • rust_hft/tools/collector/src/polymarket_evidence_artifact.rs
  • rust_hft/tools/collector/src/polymarket_research_import.rs
  • rust_hft/tools/collector/src/polymarket_research_normalize.rs
📝 Walkthrough

Walkthrough

Polymarket collection now emits event-local trade completion proofs. Upload, import, normalization, and evidence sealing validate completion metadata against canonical trade record IDs, counts, digests, market identity, and timing. Mission snapshot references and task tracking are also updated.

Changes

Trade completion proof pipeline

Layer / File(s) Summary
Collector completion emission and recovery
rust_hft/tools/collector/src/polymarket_raw.rs
The collector emits versioned completion records containing trade counts and record-ID digests, recognizes them during replay, and resets stale completion state during migration.
Completion validation and manifest import
rust_hft/tools/collector/src/polymarket_upload.rs, rust_hft/tools/collector/src/polymarket_research_import.rs
Tape scanning validates completion payloads and writes trade_completions into manifests; reference import deserializes and validates the new completion contract.
Trade binding in normalization and sealed evidence
rust_hft/tools/collector/src/polymarket_research_normalize.rs, rust_hft/prediction-markets/crates/ploy-market-data/src/polymarket_evidence/*, rust_hft/tools/collector/src/polymarket_evidence_artifact.rs, rust_hft/prediction-markets/crates/ploy-research/src/polymarket_evidence_projection.rs
Normalization tracks per-market trade record IDs and verifies completion digests; evidence validation and NDJSON framing enforce matching market identity, counts, digests, and selected markets. Fixtures and semantics assertions are updated.
Mission snapshot and task tracking
rust_hft/prediction-markets/config/research_missions/*, rust_hft/prediction-markets/tasks/todo.md
BTC and SOL mission snapshot identifiers are changed, and the completion-proof task is marked done.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ReferenceCollector
  participant PolymarketTape
  participant TapeScanner
  participant ResearchNormalizer
  participant EvidenceSealer
  ReferenceCollector->>PolymarketTape: emit trade completion proof
  PolymarketTape->>TapeScanner: scan completion update
  TapeScanner->>ResearchNormalizer: provide trade_completions manifest
  ResearchNormalizer->>ResearchNormalizer: compare trade IDs, count, and digest
  ResearchNormalizer->>EvidenceSealer: pass validated inputs and rows
  EvidenceSealer->>EvidenceSealer: enforce completion linkage during sealing
Loading

Possibly related PRs

  • proerror77/monday#34: Introduces the collector and uploader code extended here for completion proofs.
  • proerror77/monday#96: Establishes the typed evidence normalization code extended with completion validation.
  • proerror77/monday#113: Changes trade normalization behavior that this PR further binds to completion proofs.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 23.08% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately describes the main change: binding Polymarket evidence to trade completion proofs.
Description check ✅ Passed The description follows the required template and covers contract, out of scope, merge order, validation, rollout, and scope exception.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/polymarket-trade-tape-replay

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0ba046b024

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1690 to +1695
let (trade_count, trade_record_ids_sha256) =
if let Some(record_ids) = state.trade_seen.get(condition_id) {
(
record_ids.len(),
trade_record_ids_sha256(record_ids.keys().map(String::as_str)),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Scope completion proofs to the segment carrying the trades

When trades for a market are written before an hourly rotation or collector restart but the completion marker is written afterward, this hashes the cumulative state.trade_seen set, including IDs from the earlier tape. validate_event_local_trade_completions later hashes only the trade rows in the single proof-bearing reference segment, so valid boundary cohorts (for example, a 5-minute market ending at the hour) and interrupted collections are rejected as proof mismatches. The proof must reference a segment containing all hashed trades, or normalization must resolve the proof across the relevant segments.

Useful? React with 👍 / 👎.

Comment on lines +610 to +611
|| !SYMBOLS.contains(&completion.symbol.as_str())
|| completion.market_window_secs != 300

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Filter completion identities to selected evidence markets

When the reference hour also contains a completion for an unselected symbol or a 15-minute market, this rejects the entire sealed BTC/SOL five-minute artifact even though normalization intentionally emitted no rows for that market. The reference collector accepts all configured symbols and both 300/900-second markets, while normalize_raw preserves the reference segment's complete trade_completions map in validated_inputs; therefore ordinary mixed collector segments fail here on an ETH/XRP completion or any 900-second completion. Filter the map to selected contracts before publishing, or validate scope-specific fields only for completions referenced by normalized market_contract rows.

Useful? React with 👍 / 👎.

Comment on lines +1050 to +1052
let (market_id, context, completion) =
validate_trade_completion(update, sequence, recorded_at, line_number)?;
if trade_completions.insert(market_id, completion).is_some() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Verify each completion digest while scanning the raw tape

When an active or staged tape contains a well-formed completion marker whose count or digest contradicts its trade rows, validate_trade_completion checks only the marker's shape and this code copies the claim into the manifest without recomputing it. Consequently scan_tape and validate_reference_tape_for_recovery accept the false proof; recovery then marks the market trade_complete and stops polling, and the standalone research-segment validation can certify the artifact until a later selected-event normalization happens to catch it. Accumulate canonical trade IDs per market and compare their count, digest, and ordering to the marker before inserting the completion.

Useful? React with 👍 / 👎.

proerror added 2 commits July 19, 2026 18:12
…lymarket-trade-tape-replay

# Conflicts:
#	rust_hft/tools/collector/src/polymarket_upload.rs
@proerror77
proerror77 changed the base branch from main to codex/polymarket-reference-segment-set July 19, 2026 11:42
@proerror77
proerror77 changed the base branch from codex/polymarket-reference-segment-set to main July 19, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant