Skip to content

ci(rust): incremental builds, sccache, and fast-lane split - #559

Merged
proerror77 merged 1 commit into
mainfrom
codex/ci-rust-speed
Aug 1, 2026
Merged

proerror77 merged 1 commit into
mainfrom
codex/ci-rust-speed

Conversation

@proerror77

Copy link
Copy Markdown
Owner

Change contract

Rust CI gets cross-run compilation caching (sccache with GHA backend, mirroring the repo's own release-rust.yml pattern, RUSTC_WRAPPER opt-in per rust_hft/.cargo/config.toml) and a parallel "Rust Fast Gates" job (layout, Python-ban, fmt, bash -n, shellcheck, shell contract suites) that fails within ~3 minutes instead of after 10+ minutes of compilation. CARGO_INCREMENTAL stays 0 — justification: no edit-recompile cycles exist within a single CI run; across runs, sccache is the more robust reuse mechanism and incremental would bloat cache/non-determinism.

Out of scope

  • Test contents, gate standards, fail-closed behavior (all preserved; ci-gate membership retained)
  • Other workflows (release-rust/ploy-ci/acr-publish already have sccache; security-enabled is light)
  • test-polymarket-raw-ops-control-plane.sh stays in the heavy job — it requires cargo (builds hft-collector)

Dependency / merge order

None. One cold run after merge is expected (new cache keys include the sccache version).

Focused validation

  • PyYAML parse + programmatic topology check of ci.yml; bash -n on every embedded run block
  • .github/scripts/test-select-rust-ci-scope.sh suite passes (6 new contract greps pin the fast job + sccache lines)
  • Relocated run blocks verified byte-identical vs main
  • Cannot verify locally: actual wall-clock speedup — measurement plan: cold run on main after merge vs warm run on the next PR (compare Rust Workspace duration + sccache --show-stats hit rate; Fast Gates ≤3 min vs old end-of-pipeline failure point)

Rollout / rollback impact

CI-only. Rollback = revert; old caches untouched due to new key dimensions.

Issue relationship

Refs #558

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@proerror77, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 43 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6ee895c4-c9bd-43cd-be74-f0e1f9f1ce7f

📥 Commits

Reviewing files that changed from the base of the PR and between 7733a68 and 3371740.

📒 Files selected for processing (2)
  • .github/scripts/test-select-rust-ci-scope.sh
  • .github/workflows/ci.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread .github/workflows/ci.yml
@proerror77
proerror77 force-pushed the codex/ci-rust-speed branch from 82ff048 to 3371740 Compare August 1, 2026 04:22
@proerror77
proerror77 merged commit 2a22e9b into main Aug 1, 2026
42 checks passed
@proerror77
proerror77 deleted the codex/ci-rust-speed branch August 1, 2026 04:53
proerror77 pushed a commit that referenced this pull request Aug 1, 2026
- sccache-action steps are continue-on-error with an explicit fallback
  that unsets RUSTC_WRAPPER, so a flaky action cannot block the
  merge-gate path (the concern behind 92991a3's removal).
- Contract test now pins the rust_fast_gates scope condition inside its
  job block, requires sccache wiring in BOTH heavy jobs, and adds
  negative pins so moved work (fmt/shellcheck/shell suites) cannot
  silently reappear in the heavy rust job.

Refs #558
proerror77 pushed a commit that referenced this pull request Aug 1, 2026
- sccache-action steps are continue-on-error with an explicit fallback
  that unsets RUSTC_WRAPPER, so a flaky action cannot block the
  merge-gate path (the concern behind 92991a3's removal).
- Contract test now pins the rust_fast_gates scope condition inside its
  job block, requires sccache wiring in BOTH heavy jobs, and adds
  negative pins so moved work (fmt/shellcheck/shell suites) cannot
  silently reappear in the heavy rust job.

Refs #558
proerror77 added a commit that referenced this pull request Aug 1, 2026
…iew (#566)

* ci(rust): harden sccache setup and scope-contract pins after #559 review

- sccache-action steps are continue-on-error with an explicit fallback
  that unsets RUSTC_WRAPPER, so a flaky action cannot block the
  merge-gate path (the concern behind 92991a3's removal).
- Contract test now pins the rust_fast_gates scope condition inside its
  job block, requires sccache wiring in BOTH heavy jobs, and adds
  negative pins so moved work (fmt/shellcheck/shell suites) cannot
  silently reappear in the heavy rust job.

Refs #558

* test(ci): per-job positive and negative scope pins, verified by mutation

Second-round review: the earlier pins were file-wide counts with a
leaky awk block and no positive placement assertions. Now each job
block is extracted exactly, sccache presence is asserted per job,
suite placement is pinned both directions (including the
market-recorder suite's home in the heavy job), and a mutation test
(recorder moved back to fast gates) proves the pins fail.

Refs #558

---------

Co-authored-by: Sonic Shih <sonic.shih@mandonothing.com>
proerror77 pushed a commit that referenced this pull request Aug 1, 2026
Replace the homegrown sccache setup (apt/cargo install plus actions/cache
on ~/.cache/sccache, whose research-image primary key embedded github.sha
and could never hit) with the #559/#566 pattern in every job that compiles
Rust on the runner:

- mozilla-actions/sccache-action@v0.0.10 with continue-on-error and a
  fallback step that unsets RUSTC_WRAPPER/SCCACHE_GHA_ENABLED
- job env RUSTC_WRAPPER=sccache and SCCACHE_GHA_ENABLED=true
- Swatinem/rust-cache keys dimensioned on rustc version, sccache
  version, and Cargo.lock
- ${SCCACHE_PATH:-sccache} --show-stats || true evidence steps

The acr-publish publish matrix compiles inside docker buildx, so the
host-side wrapper does not apply there and that job is unchanged. No
test semantics, job conditions, or gating changed.

Refs #567
proerror77 added a commit that referenced this pull request Aug 1, 2026
#570)

* ci(rust): backfill sccache-action pattern onto ploy-ci and acr-publish

Replace the homegrown sccache setup (apt/cargo install plus actions/cache
on ~/.cache/sccache, whose research-image primary key embedded github.sha
and could never hit) with the #559/#566 pattern in every job that compiles
Rust on the runner:

- mozilla-actions/sccache-action@v0.0.10 with continue-on-error and a
  fallback step that unsets RUSTC_WRAPPER/SCCACHE_GHA_ENABLED
- job env RUSTC_WRAPPER=sccache and SCCACHE_GHA_ENABLED=true
- Swatinem/rust-cache keys dimensioned on rustc version, sccache
  version, and Cargo.lock
- ${SCCACHE_PATH:-sccache} --show-stats || true evidence steps

The acr-publish publish matrix compiles inside docker buildx, so the
host-side wrapper does not apply there and that job is unchanged. No
test semantics, job conditions, or gating changed.

Refs #567

* ci(rust): tolerate missing sccache binary in cache dimension steps

CodeRabbit P1 on #570: when sccache setup fails (the case
continue-on-error tolerates), the unconditional version lookup would
fail the job. Mirrors the ci.yml fix.

Refs #567

---------

Co-authored-by: Sonic Shih <sonic.shih@mandonothing.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants