Skip to content

fix(data): validate market-tape receive clocks per stream - #155

Merged
proerror77 merged 1 commit into
mainfrom
codex/binance-tape-stream-receive-clock
Jul 19, 2026
Merged

proerror77 merged 1 commit into
mainfrom
codex/binance-tape-stream-receive-clock

Conversation

@proerror77

@proerror77 proerror77 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Change contract
Validate Binance market-tape receive timestamps independently for each symbol depth stream and aggregate-trade stream, so legitimate concurrent cross-stream append interleaving is accepted while same-stream and cross-segment rollback remain fail-closed.

Out of scope
Collector changes, artifact rewriting, timestamp tolerance, Polymarket evidence, snapshot/evaluator behavior, full-tick research, deployment, and live trading.

Dependency or merge order
None. This is a focused verifier correction on current main.

Focused validation

  • Cloud exact-main reproduction: BTC/SOL spot segment failed on an 81ns cross-stream interleave: SOL aggregate trade 1784380149716383136, followed by BTC depth 1784380149716383055.
  • Added the matching two-symbol acceptance counterexample.
  • Existing cross-segment depth receive rollback counterexample remains fail-closed.
  • Added a backdated session_start counterexample so non-stream rows cannot bypass the segment/session boundary.
  • cargo fmt --package hft-data
  • git diff --check
  • Linux CI is the execution authority; no local build/test was run.

Rollout or rollback impact
No collector or runtime rollout. Research verification will accept only cross-stream receive-time interleaving; digest, manifest, session, segment-envelope, source-clock, sequence, replay, and per-stream receive-clock checks remain fail-closed. Rollback restores the overly global clock gate and makes the existing immutable tape ineligible for research.

@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

verify_binance_market_tape now enforces receive-time monotonicity independently by symbol and event stream, allowing independent streams to interleave. A regression test covers interleaved trade and checkpoint receive times.

Changes

Binance receive-clock validation

Layer / File(s) Summary
Per-stream clock state
rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
Per-symbol depth and aggregate-trade receive clocks use observe_receive_clock to reject backward timestamps within the same stream.
Verification integration and regression coverage
rust_hft/data-pipelines/core/src/binance_market_tape_artifact.rs
Diff, snapshot, checkpoint, and aggregate-trade rows use their respective clocks; a test verifies that independent streams may interleave receive times.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • proerror77/monday#104: Also changes Binance market-tape receive-time ordering and clock invariants.
  • proerror77/monday#131: Also modifies verify_binance_market_tape receive-time tracking and per-symbol replay behavior.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the verifier change to validate market-tape receive clocks per stream.
Description check ✅ Passed The description covers the main template sections and includes the contract, out-of-scope, validation, and rollback details.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/binance-tape-stream-receive-clock

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proerror77
proerror77 force-pushed the codex/binance-tape-stream-receive-clock branch from f8f6851 to 07cb528 Compare July 19, 2026 11:10
@proerror77
proerror77 merged commit c487019 into main Jul 19, 2026
17 checks passed
@proerror77
proerror77 deleted the codex/binance-tape-stream-receive-clock branch July 19, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant