Skip to content

fix(collector): bound strict gate verifier memory - #359

Merged
proerror77 merged 5 commits into
mainfrom
codex/collector-gate-bounded-verifier-224
Jul 26, 2026
Merged

proerror77 merged 5 commits into
mainfrom
codex/collector-gate-bounded-verifier-224

Conversation

@proerror77

@proerror77 proerror77 commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Change contract

Make the #224 strict shadow gate bounded and fail-closed: verify adjacent segment pairs with transient systemd cgroups, clean those units on failure/termination, and use a Rust gate-only verifier that keeps all SHA, _SUCCESS, agg_trade, manifest, and LOB-continuity validation while not retaining research surfaces.

PRD

Not required; this is a narrow regression fix for the existing #224 collector contract.

Out of scope

  • Binance source collector acquisition/subscription behavior
  • Snapshot or research logic
  • Historical 7/20 data repair
  • Production service cutover
  • Any Polymarket changes

Dependency / merge order

None. Based on main at 1f40d29. Issue #224.

Focused validation

  • bash deployment/aliyun/test-rust-lob-control-plane.sh
  • bash -n deployment/aliyun/host-rust-lob-shadow-gate.sh
  • shellcheck deployment/aliyun/host-rust-lob-shadow-gate.sh deployment/aliyun/test-rust-lob-control-plane.sh
  • git diff --check
  • rustfmt --check rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
  • cargo test -p hft-data --lib (52 passed)
  • cargo test -p hft-collector --bin binance-lob-archiver (63 passed)
  • Counterexample fixtures cover adjacent-pair invocation and transient-unit cleanup after verifier failure.

Rollout / rollback impact

Release candidate only: merge/CI builds a new artifact, then install it as an inactive ECS shadow candidate and rerun the bounded gate. No production service was changed by this PR. Roll back by retaining the previous candidate/artifact; cutover remains a separate, explicitly controlled operation after gate evidence and canonical readback.

Summary by CodeRabbit

  • New Features

    • Added strict verification for market-tape segments using adjacent-segment checks.
    • Added resource controls and automatic cleanup for verification processes.
    • Added a dedicated verification mode that avoids collecting unnecessary replay data.
    • Updated segment verification to use the strict verifier when continuity checks are required.
  • Bug Fixes

    • Improved failure handling by stopping in-progress verification tasks when errors occur.
    • Corrected verified segment counts reported after successful checks.
  • Tests

    • Expanded coverage for adjacent-pair verification, resource limits, command arguments, and failure cleanup.

@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@proerror77, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 14 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 033e2749-5cd8-4e07-be1e-16ede1f29b11

📥 Commits

Reviewing files that changed from the base of the PR and between 64e0766 and 92601a9.

📒 Files selected for processing (4)
  • deployment/aliyun/host-rust-lob-shadow-gate.sh
  • deployment/aliyun/test-rust-lob-control-plane.sh
  • rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
  • rust_hft/tools/collector/src/bin/binance-lob-archiver.rs
📝 Walkthrough

Walkthrough

The PR adds a surface-free strict tape verifier, routes strict LOB continuity checks through it, and changes the gate to verify adjacent segment pairs in bounded systemd-run units with cleanup and failure handling.

Changes

Strict gate verification

Layer / File(s) Summary
Strict tape verifier contract
rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
Adds verify_binance_market_tape_for_strict_gate, conditionally skips replay-surface collection, and tests complete V1 tape verification without collected surfaces.
Collector strict verification wiring
rust_hft/tools/collector/src/bin/binance-lob-archiver.rs
Selects the strict-gate verifier when LOB continuity is required and reports the number of input sealed segments.
Adjacent-pair gate orchestration
deployment/aliyun/host-rust-lob-shadow-gate.sh, deployment/aliyun/test-rust-lob-control-plane.sh
Runs strict verification for adjacent segment pairs in bounded transient units, tracks and stops failed units, accumulates segment tuples, and validates success and failure fixtures.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ShadowGate
  participant SystemdRun
  participant Archiver
  participant StrictVerifier

  ShadowGate->>SystemdRun: Start bounded pair unit
  SystemdRun->>Archiver: Run --verify-segment with continuity
  Archiver->>StrictVerifier: Verify strict market tape
  StrictVerifier-->>Archiver: Return verification result
  Archiver-->>SystemdRun: Return pair status
  SystemdRun-->>ShadowGate: Return unit status
  ShadowGate->>SystemdRun: Stop failed unit when needed
Loading

Possibly related PRs

  • proerror77/monday#131: Extends the same Binance tape verifier with replay timeline and received-time data.
  • proerror77/monday#265: Modifies the same shadow gate’s strict-verifier arguments and LOB continuity checks.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: bounding the strict gate verifier's memory.
Description check ✅ Passed The description matches the template and covers contract, out-of-scope, dependencies, validation, and rollback; Scope exception should explicitly say None.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/collector-gate-bounded-verifier-224

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64e07661c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread deployment/aliyun/host-rust-lob-shadow-gate.sh
Comment thread deployment/aliyun/host-rust-lob-shadow-gate.sh Outdated
Comment thread deployment/aliyun/host-rust-lob-shadow-gate.sh

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs (1)

204-268: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Strict-gate surface-skipping design looks correct.

Correctness-critical checks (replay sequencing, aggregate-trade sequencing/summaries, LOB continuity builder, non-empty/positive book levels) all still run unconditionally; only surface materialization is skipped when collect_surfaces is false. This matches the PR's stated goal of bounding memory without weakening validation for the strict gate.

One latent footgun worth guarding: the unchanged block further down (around the !require_lob_continuity branch that inspects aggregate_trades) assumes aggregate_trades reflects the tape's real trades. Since aggregate_trades.push is now gated behind collect_surfaces (see the "agg_trade" handling below), a future caller of this private helper with (require_lob_continuity=false, collect_surfaces=false) would always see an empty aggregate_trades, causing that branch to reject every tape as "missing aggregate trades for a declared symbol" regardless of actual content. Today this is unreachable (the only collect_surfaces=false caller always sets require_lob_continuity=true), but nothing enforces that pairing.

🛡️ Suggested guard
 fn verify_binance_market_tape_with_requirements_and_surfaces(
     mut sealed: Vec<SealedBinanceMarketTapeTriplet>,
     require_trade_summaries: bool,
     require_lob_continuity: bool,
     collect_surfaces: bool,
 ) -> Result<VerifiedBinanceMarketTape> {
+    debug_assert!(
+        collect_surfaces || require_lob_continuity,
+        "skipping surface collection is only safe when LOB continuity is required, \
+         since the non-continuity path relies on collected aggregate trades"
+    );
     if sealed.is_empty() {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs` around
lines 204 - 268, Guard the parameter combination in
verify_binance_market_tape_with_requirements_and_surfaces so
collect_surfaces=false cannot be used with require_lob_continuity=false, or
otherwise preserve aggregate-trade availability for that validation branch.
Ensure the existing strict-gate call remains valid and prevent future callers
from reaching the !require_lob_continuity aggregate_trades check with an
intentionally unmaterialized surface.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@deployment/aliyun/host-rust-lob-shadow-gate.sh`:
- Around line 269-287: Update the verification flow around
verify_oss_round_trips so it does not run via command substitution or another
subshell when assigning round_trips. Use an output-variable, file, or pipe-based
pattern that preserves the function’s result while keeping mutations to
strict_verifier_unit in the parent shell, ensuring cleanup can stop any unit
created by run_strict_verifier_pair.

---

Nitpick comments:
In `@rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs`:
- Around line 204-268: Guard the parameter combination in
verify_binance_market_tape_with_requirements_and_surfaces so
collect_surfaces=false cannot be used with require_lob_continuity=false, or
otherwise preserve aggregate-trade availability for that validation branch.
Ensure the existing strict-gate call remains valid and prevent future callers
from reaching the !require_lob_continuity aggregate_trades check with an
intentionally unmaterialized surface.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 726bae98-864b-4cbf-96c2-32679f02b6be

📥 Commits

Reviewing files that changed from the base of the PR and between 1f40d29 and 64e0766.

📒 Files selected for processing (4)
  • deployment/aliyun/host-rust-lob-shadow-gate.sh
  • deployment/aliyun/test-rust-lob-control-plane.sh
  • rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
  • rust_hft/tools/collector/src/bin/binance-lob-archiver.rs

Comment thread deployment/aliyun/host-rust-lob-shadow-gate.sh
@proerror77

Copy link
Copy Markdown
Owner Author

Addressed the review findings in commit 574b5e22: verify_oss_round_trips now runs in the parent shell and writes JSON to a temp file before readback, so transient verifier state remains visible to the parent cleanup trap; the Rust surface-free verifier now fail-closes on an invalid non-LOB mode and has a regression test. Focused local validation remains green: hft-data 53/53, collector 63/63, control-plane contract, shellcheck, rustfmt, and diff check.

@proerror77

Copy link
Copy Markdown
Owner Author

Addressed the remaining review findings in d327202:

  • Added a bounded, one-segment-at-a-time aggregate-trade continuity pass. It carries per-symbol aggregate ID, source-clock, and received-clock state across the full ordered segment set, so a symbol absent from an intermediate segment cannot reset continuity.
  • Kept the existing adjacent-pair strict verifier for per-segment SHA/manifest/_SUCCESS and LOB replay/continuity checks.
  • Raised only the transient strict-verifier cgroup to MemoryHigh=5000M / MemoryMax=6400M, covering two accepted 2 GiB decompressed + 512 MiB compressed segment bounds while keeping the verifier isolated and surface-free.

Validation: shell syntax + shellcheck + control-plane contract test; hft-data 54/54; hft-collector binance-lob-archiver 63/63; rustfmt check; git diff --check.

@proerror77

Copy link
Copy Markdown
Owner Author

Root cause fixed in 4f9c6d1: the full-set aggregate continuity gate passed bare path/SHA/manifest triples to the collector CLI instead of reconstructing the required --verify-segment / --segment-content-sha256 / --segment-manifest-sha256 flags. macOS Bash 3.2 masked the failing fixture via set -e behavior; GitHub Bash 5 failed silently. The fixture now fails explicitly and verifies all three isolated verifier invocations. Validation: Bash 5 and Bash 3.2 control-plane tests pass; exact CI release-control command passes; shellcheck and git diff --check pass; two-axis incremental review is clean.

@proerror77
proerror77 merged commit f057e25 into main Jul 26, 2026
38 checks passed
@proerror77
proerror77 deleted the codex/collector-gate-bounded-verifier-224 branch July 26, 2026 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant