fix(data): honor Binance USD-M pu continuity

[proerror77]

Change contract

Accept Binance USD-M diff-depth frames when pu exactly matches the previous event's u, even when U is nonconsecutive, while preserving fail-closed no-pu adjacency, previous-ID mismatch, final-ID rollback, and source-clock checks.

Why

An isolated ECS capture on the production host reproduced the failure with consecutive authenticated frames: previous u=11074967403842; next pu=11074967403842, U=11074967403847, u=11074967407986. The existing validator rejected the second frame solely because U > previous_u + 1, although Binance USD-M continuity is defined by pu == previous u.

Official contract: https://developers.binance.com/en/docs/products/derivatives-trading-usds-futures/websocket-market-streams/How-to-manage-a-local-order-book-correctly

Out of scope

No collector retry/lifecycle changes, artifact schema or trust-anchor changes, evaluator/snapshot work, deployment/cutover changes, production service changes, or live-trading enablement.

Dependency / merge order

None. Based directly on main after #120. Merge this PR before rebuilding and rerunning the isolated ECS Binance candidate.

Focused validation

• Red before fix: exact ECS-shaped pu/u/U counterexample failed with depth sequence gap expected=11074967403843 received=11074967403847.
• Green after fix: the same focused test passes.
• cargo test -p hft-data: 23 passed.
• cargo test -p hft-collector --bin binance-lob-archiver: 36 passed.
• cargo clippy -p hft-data --all-targets -- -D warnings: passed.
• cargo clippy -p hft-collector --bin binance-lob-archiver -- -D warnings: passed.
• Target file rustfmt --check: passed.
• Exact-spec and code review: no findings; existing tests retain pu gap/rollback, no-pu gap, final-ID rollback, and source-time counterexamples.

The repository-wide cargo fmt --all -- --check still reports unrelated pre-existing formatting drift outside this PR; no unrelated files were reformatted.

Rollout / rollback impact

No automatic deployment. After merge, publish a new immutable Linux collector bundle, rerun the isolated BTC/SOL ECS capture, and require zero sequence gaps before using the tape. Roll back by reverting this commit; the current production collectors remain untouched and live trading remains disabled.

Summary by CodeRabbit

• Bug Fixes
  • Improved depth update sequence validation for futures market data.
  • Correctly accepts valid continuity when updates provide a matching previous update ID, even if the next update ID is nonconsecutive.
  • Preserved detection of sequence gaps and rollbacks.

[coderabbitai]

📝 Walkthrough

Walkthrough

The depth sequence validator now computes the expected update ID before handling previous final update IDs. A unit test verifies that futures depth continuity is accepted when the next first update ID is non-consecutive but its previous final update ID establishes continuity.

Changes

Depth sequence validation

Layer / File(s)	Summary
Validator ordering and continuity coverage rust_hft/data-pipelines/core/src/binance_market_tape.rs	DepthSourceClockSequenceValidator::observe computes the expected update ID before previous_final_update_id checks, and a new test covers continuity with a non-consecutive next first_update_id.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• proerror77/monday#104: Introduced related Binance market-tape sequence validation logic used by this refactor and test.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the Binance USD-M pu continuity fix.
Description check	✅ Passed	The description is mostly complete and covers the behavior change, rationale, validation, and rollout; the scope-exception line is not explicitly filled in.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/binance-usdm-pu-continuity

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust_hft/data-pipelines/core/src/binance_market_tape.rs`:
- Around line 109-119: Extend DepthDiff to retain previous_final_update_id from
each Binance depth event, and populate it when constructing diffs in the
market-tape pipeline. Update LocalBook::apply_diff to use this preserved value
when validating non-consecutive update IDs, allowing valid Binance bridge diffs
while still rejecting genuine gaps.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: daf5f579-798b-4605-90e1-3b09fed8e713

📥 Commits

Reviewing files that changed from the base of the PR and between 25cb490 and d77d70a.

📒 Files selected for processing (1)

• rust_hft/data-pipelines/core/src/binance_market_tape.rs