Skip to content

fix(collector): allow ossutil state cache writes - #13

Merged
proerror77 merged 1 commit into
mainfrom
codex/collector-oss-cache-fix
Jul 14, 2026
Merged

proerror77 merged 1 commit into
mainfrom
codex/collector-oss-cache-fix

Conversation

@proerror77

@proerror77 proerror77 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • let systemd manage /var/lib/hft-collector as the collector state directory
  • preserve ProtectSystem=strict and the existing spool-only write allowance
  • restore Aliyun ossutil version-cache writes and queued OSS uploads

Verification

  • systemd-analyze verify passed on monday-trade-data-1
  • spot backlog: 5 segments uploaded, pending manifests 0, health synced, gaps 0
  • USDM backlog: 4 segments uploaded, pending manifests 0, OSS _SUCCESS through hour 11
  • standards review: pass (0 findings)
  • spec review: pass (0 findings)

Summary by CodeRabbit

  • Bug Fixes
    • Improved service deployment reliability by ensuring the required application state directory is created and managed automatically.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The templated Binance LOB archiver systemd unit now declares hft-collector as its managed state directory.

Changes

Service state configuration

Layer / File(s) Summary
Declare archiver state directory
deployment/aliyun/binance-lob-archiver@.service
Adds StateDirectory=hft-collector under the [Service] section.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: enabling ossutil state cache writes for the collector.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/collector-oss-cache-fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@deployment/aliyun/binance-lob-archiver`@.service:
- Line 11: Update the service’s environment mapping to set SPOOL_DIR under
${STATE_DIRECTORY}, replacing the hard-coded /data/monday/spool/binance-lob path
for the archiver and ossutil cache. Keep StateDirectory=hft-collector and ensure
both spot and usdm spool paths use the systemd-managed persistent state
directory.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: dd666b28-afe3-4eb3-84f1-205df4b8cc74

📥 Commits

Reviewing files that changed from the base of the PR and between be1cd98 and 828a421.

📒 Files selected for processing (1)
  • deployment/aliyun/binance-lob-archiver@.service

Type=simple
User=hftcollector
Group=hftcollector
StateDirectory=hft-collector

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

rg -n 'STATE_DIRECTORY|SPOOL_DIR|ossutil|cache|queue' deployment rust_hft \
  --glob '!**/.git/**'

rg -n 'EnvironmentFile|Environment=' deployment/aliyun/binance-lob-archiver@.service

Repository: proerror77/monday

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -eu

printf '\n## service\n'
cat -n deployment/aliyun/binance-lob-archiver@.service

printf '\n## env files in deployment/aliyun matching archiver\n'
fd -a 'binance-lob-archiver*.env' deployment/aliyun

printf '\n## spot env\n'
cat -n deployment/aliyun/binance-lob-archiver-spot.env

printf '\n## references to STATE_DIRECTORY in deployment/aliyun and rust_hft/tools/collector\n'
rg -n 'STATE_DIRECTORY|SPOOL_DIR|COLLECTOR_SPOOL_DIR|ossutil' deployment/aliyun rust_hft/tools/collector --glob '!**/.git/**'

Repository: proerror77/monday

Length of output: 4367


🏁 Script executed:

#!/bin/bash
set -eu

printf '\n## service file\n'
cat -n deployment/aliyun/binance-lob-archiver@.service

printf '\n## archiver env files\n'
for f in deployment/aliyun/binance-lob-archiver*.env; do
  echo "== $f =="
  cat -n "$f"
done

printf '\n## archiver implementation references\n'
rg -n 'SPOOL_DIR|STATE_DIRECTORY|health\.json|queued|ossutil' deployment/aliyun rust_hft/tools/collector --glob '!**/.git/**' | head -n 120

Repository: proerror77/monday

Length of output: 5292


Wire SPOOL_DIR to ${STATE_DIRECTORY} in the env mapping.
StateDirectory=hft-collector only creates /var/lib/hft-collector; the service still loads SPOOL_DIR=/data/monday/spool/binance-lob/{spot,usdm} from the env files, so the systemd state dir remains unused. Map the archiver/ossutil state path through ${STATE_DIRECTORY} if this setting is meant to provide the persistent cache.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@deployment/aliyun/binance-lob-archiver`@.service at line 11, Update the
service’s environment mapping to set SPOOL_DIR under ${STATE_DIRECTORY},
replacing the hard-coded /data/monday/spool/binance-lob path for the archiver
and ossutil cache. Keep StateDirectory=hft-collector and ensure both spot and
usdm spool paths use the systemd-managed persistent state directory.

@proerror77
proerror77 merged commit 582e145 into main Jul 14, 2026
28 of 30 checks passed
@proerror77
proerror77 deleted the codex/collector-oss-cache-fix branch July 14, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant