Parallel frontend by default

[zetanumbers]

View all comments

I would like to help push progress on parallel compiler. I think it will be necessary at some point to test compiler in its multi-threaded mode (we already "have" parallel compiler but it is single-threaded by default) via crater. It would allow us to diagnose new issues, measure impact on the ecosystem and estimate amount of remaining work.

I have modified compiler to use "num_cpus" number of 8 threads by default. But I am not sure if there's better value for the cloud environment, so please tell if there is one. I am also not aware of any other nuances of cloud computing, but @Mark-Simulacrum on zulip have confirmed it should be fine.

I believe build-and-test crater mode should provide us with more useful information than the other modes, but if necessary you can scale it down to just build only.

Related zulip thread: #t-compiler/parallel-rustc > Run crater with `-Zthreads=$NUM_CPUS` set

EDIT: Switched from num_cpus to 8 threads

[rustbot]

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[lqd]

@bors try

[bjorn3]

@bors try spurious failure

[rust-bors]

Unknown argument "spurious". Run @bors2 help to see available commands.

[bjorn3]

Never mind, that was PR CI, not the try job.

[rust-bors]

☀️ Try build successful (CI)
Build commit: d79d87d (d79d87d4c0f7f0d0785ad6e493cf8cf718ed89a6, parent: ad85bc524b1ad696e42061ad8338d382dffbdbe5)

[petrochenkov]

@craterbot build-and-test

[craterbot]

🚨 Error: failed to parse the command

🆘 If you have any trouble with Crater please ask in t-infra on Zulip
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[petrochenkov]

@craterbot run mode=build-and-test

[craterbot]

👌 Experiment pr-146237 created and queued.
🤖 Automatically detected try build d79d87d
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-146237 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-146237 is completed!
📊 182 regressed and 154 fixed (694485 total)
📊 115635 spurious results on the retry-regessed-list.txt, consider a retry¹ if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-146237/retry-regressed-list.txt↩

[zetanumbers]

OOM build failures increasing from 4.5% to 13.5% is concerning if we will have more that one thread enabled by default one day. I hypothesized in the zulip thread it comes from increased RAM consumption during parallel compilation. I guess if we set threads to num_cpus (which is our eventual goal) this problem would become less noticeable. So I have reverted to this behavior to see what happens now.

@craterbot run mode=build-only crates=https://crater-reports.s3.amazonaws.com/pr-146237/retry-regressed-list.txt

[craterbot]

🔒 Error: you're not allowed to interact with this bot.

🔑 If you are a member of a Rust team and need access, please update rust-lang/team to grant your team or yourself access to the crater permission.
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[SparrowLii]

@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: 4987e9d (4987e9d4227139400384ab59296ffee3b0fb1183, parent: c7f6aa2869acdbf014d094c6e427e554e160b6db)

[SparrowLii]

@craterbot run mode=build-only crates=https://crater-reports.s3.amazonaws.com/pr-146237/retry-regressed-list.txt

[craterbot]

👌 Experiment pr-146237-2 created and queued.
🤖 Automatically detected try build 4987e9d
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[SparrowLii]

@craterbot p=1

[craterbot]

📝 Configuration of the pr-146237-2 experiment changed.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-146237-2 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-146237-2 is completed!
📊 3 regressed and 1 fixed (115430 total)
📊 32026 spurious results on the retry-regessed-list.txt, consider a retry¹ if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-146237-2/retry-regressed-list.txt↩

[SpriteOvO]

Hi, ping from triage team. This PR has been inactive for a while. What's the status of this PR now? Thanks.

[zetanumbers]

Hi, ping from triage team. This PR has been inactive for a while. What's the status of this PR now? Thanks.

I plan to rerun crater to see if I am right about zero regressions after #149849 is merged and other small stuff is done. See #t-compiler/parallel-rustc > Steps to enable/stabilize multi-threaded rustc front-end @ 💬

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[zetanumbers]

This is it. Let's merge this PR and get over with it.

[rust-log-analyzer]

The job aarch64-gnu-llvm-21-1 failed! Check out the build log: (web)(plain enhanced)(plain)

Click to see the possible cause of the failure (guessed by this bot)

Executing "/scripts/stage_2_test_set1.sh"
+ /scripts/stage_2_test_set1.sh
PR_CI_JOB set; skipping tidy
+ '[' 1 == 1 ']'
+ echo 'PR_CI_JOB set; skipping tidy'
+ SKIP_TIDY='--skip tidy'
+ ../x.py --stage 2 test --skip tidy --skip compiler --skip src
##[group]Building bootstrap
Finished `dev` profile [unoptimized] target(s) in 0.04s
##[endgroup]
downloading https://static.rust-lang.org/dist/2026-04-14/rustfmt-nightly-aarch64-unknown-linux-gnu.tar.xz
---
failures:
---- [ui] tests/ui-fulldeps/obtain-borrowck.rs stdout ----
Saved the actual run.stdout to `/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui-fulldeps/obtain-borrowck/obtain-borrowck.run.stdout`
diff of run.stdout:
1 Bodies retrieved for:
2 ::X::provided
- ::foo
- ::main
- ::main::{constant#0}
6 ::with_nested_body
7 ::with_nested_body::{closure#0}
8 ::{impl#0}::new
The actual run.stdout differed from the expected run.stdout
error: 1 errors occurred comparing run output.
status: exit status: 0
command: cd "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui-fulldeps/obtain-borrowck" && RUSTC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" RUST_TEST_THREADS="4" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/ui-fulldeps/obtain-borrowck/a" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--edition=2021" "/checkout/tests/ui-fulldeps/auxiliary/obtain-borrowck-input.rs"
--- stdout -------------------------------
Bodies retrieved for:
::X::provided
::with_nested_body
::with_nested_body::{closure#0}
::{impl#0}::new
::{impl#1}::provided
::{impl#1}::required
------------------------------------------
stderr: none
---- [ui] tests/ui-fulldeps/obtain-borrowck.rs stdout end ----

[rust-bors]

☔ The latest upstream changes (presumably #156290) made this pull request unmergeable. Please resolve the merge conflicts.

[petrochenkov]

This PR was originally used for a crater run with enabled parallel frontend.
The stabilization will happen in accordance with the MCP rust-lang/compiler-team#1005.