Skip to content

Make bootstrap self test parallel - #146155

Merged
bors merged 3 commits into
rust-lang:masterfrom
bit-aloo:2025-09-03-make-test-run-parallel
Sep 4, 2025
Merged

Make bootstrap self test parallel#146155
bors merged 3 commits into
rust-lang:masterfrom
bit-aloo:2025-09-03-make-test-run-parallel

Conversation

@bit-aloo

@bit-aloobit-aloo commented Sep 3, 2025

Copy link
Copy Markdown
Member

This PR refactors the bootstrap self test:

  1. Replaced the hardcoded --test-threads=1 with std::thread::available_parallelism(), allowing tests to run with the maximum concurrency supported by the system.
  2. Replaced custom hacks around OUT_DIR with TempDir and introduced TestCtx to simplify configuration setup.

r? @Kobzol

@bit-aloo
refactor tests to use TestCtx
df2caed
@bit-aloo
Make test parallel
c4a5408
 Use std::thread::available_parallelism() instead of hardcoded
--test-threads=1 in bootstrap tests.
@rustbotrustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Sep 3, 2025
Kobzol
Kobzol reviewed Sep 3, 2025
View reviewed changes

@KobzolKobzol left a comment •
edited by rustbot
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love how fast are the tests now, I can run all snapshot tests in under 5 seconds ❤️ Thank you!

View changes since this review

Comment threadsrc/bootstrap/src/core/build_steps/test.rs Outdated
@Kobzol

Kobzol commented Sep 3, 2025

Copy link
Copy Markdown
Member

Eventually it would be great to get rid of all usages of Config::parse from tests, except for the one in TestCtx.

@Kobzol

Kobzol commented Sep 3, 2025

Copy link
Copy Markdown
Member

Thanks! You can r=me once CI is green.

@bors delegate+

@bors

bors commented Sep 3, 2025

Copy link
Copy Markdown
Collaborator

✌️ @Shourya742, you can now approve this pull request!

If @Kobzol told you to "r=me" after making some further change, please make that change, then do @bors r=@Kobzol

@bit-aloo

bit-aloo commented Sep 3, 2025

Copy link
Copy Markdown
MemberAuthor

@bors r=@Kobzol

@bors

bors commented Sep 3, 2025

Copy link
Copy Markdown
Collaborator

📌 Commit 9f3fe40 has been approved by Kobzol

It is now in the queue for this repository.

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 3, 2025
klensy
klensy reviewed Sep 3, 2025
View reviewed changes
Comment threadsrc/bootstrap/src/core/build_steps/test.rs
Comment on lines -3297 to -3298
// bootstrap tests are racy on directory creation so just run them one at a time.
// Since there's not many this shouldn't be a problem.

@klensyklensySep 3, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implies that tests no longer racy?

@KobzolKobzolSep 3, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have attempted to remove all racy tests. If we find any more on CI, we should fix them, rather than making the test execution serial.

@ZalatharZalathar mentioned this pull request Sep 4, 2025
Merged
bors added a commit that referenced this pull request Sep 4, 2025
@bors
Auto merge of#146185- Zalathar:rollup-n14lyv6, r=Zalathar
033c0a4
Rollup of 24 pull requests
Successful merges:
- #140459 (Add `read_buf` equivalents for positioned reads)
- #143725 (core: add Peekable::next_if_map)
- #145209 (Stabilize `path_add_extension`)
- #145342 (fix drop scope for `super let` bindings within `if let`)
- #145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- #145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- #145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- #145962 (Ensure we emit an allocator shim when only some crate types need one)
- #145963 (Add LSX accelerated implementation for source file analysis)
- #146054 (add `#[must_use]` to `array::repeat`)
- #146090 (Derive `PartialEq` for `InvisibleOrigin`)
- #146112 (don't uppercase error messages)
- #146120 (Correct typo in `rustc_errors` comment)
- #146124 (Test `rustc-dev` in `distcheck`)
- #146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- #146131 (rustdoc-search: add test case for indexing every item type)
- #146134 (llvm: nvptx: Layout update to match LLVM)
- #146136 (docs(std): add missing closing code block fences in doc comments)
- #146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- #146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- #146150 (fix(rustdoc): match rustc `--emit` precedence )
- #146155 (Make bootstrap self test parallel)
- #146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- #146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
@bors
bors merged commit 8b2a64f into rust-lang:masterSep 4, 2025
10 checks passed
@rustbotrustbot added this to the 1.91.0 milestone Sep 4, 2025
rust-timer added a commit that referenced this pull request Sep 4, 2025
@rust-timer
Unrolled build for#146155
e2682dc
Rollup merge of #146155 - Shourya742:2025-09-03-make-test-run-parallel, r=Kobzol
Make bootstrap self test parallel
This PR refactors the bootstrap self test:
1. Replaced the hardcoded --test-threads=1 with std::thread::available_parallelism(), allowing tests to run with the maximum concurrency supported by the system.
2. Replaced custom hacks around OUT_DIR with TempDir and introduced TestCtx to simplify configuration setup.
r? `@Kobzol`
github-actionsBot pushed a commit to rust-lang/miri that referenced this pull request Sep 5, 2025
@bors
Auto merge of #146185 - Zalathar:rollup-n14lyv6, r=Zalathar
30def9b
Rollup of 24 pull requests
Successful merges:
- rust-lang/rust#140459 (Add `read_buf` equivalents for positioned reads)
- rust-lang/rust#143725 (core: add Peekable::next_if_map)
- rust-lang/rust#145209 (Stabilize `path_add_extension`)
- rust-lang/rust#145342 (fix drop scope for `super let` bindings within `if let`)
- rust-lang/rust#145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- rust-lang/rust#145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- rust-lang/rust#145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- rust-lang/rust#145962 (Ensure we emit an allocator shim when only some crate types need one)
- rust-lang/rust#145963 (Add LSX accelerated implementation for source file analysis)
- rust-lang/rust#146054 (add `#[must_use]` to `array::repeat`)
- rust-lang/rust#146090 (Derive `PartialEq` for `InvisibleOrigin`)
- rust-lang/rust#146112 (don't uppercase error messages)
- rust-lang/rust#146120 (Correct typo in `rustc_errors` comment)
- rust-lang/rust#146124 (Test `rustc-dev` in `distcheck`)
- rust-lang/rust#146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- rust-lang/rust#146131 (rustdoc-search: add test case for indexing every item type)
- rust-lang/rust#146134 (llvm: nvptx: Layout update to match LLVM)
- rust-lang/rust#146136 (docs(std): add missing closing code block fences in doc comments)
- rust-lang/rust#146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- rust-lang/rust#146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- rust-lang/rust#146150 (fix(rustdoc): match rustc `--emit` precedence )
- rust-lang/rust#146155 (Make bootstrap self test parallel)
- rust-lang/rust#146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- rust-lang/rust#146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
github-actionsBot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Sep 8, 2025
@bors
Auto merge of #146185 - Zalathar:rollup-n14lyv6, r=Zalathar
3d5fcdc
Rollup of 24 pull requests
Successful merges:
- rust-lang/rust#140459 (Add `read_buf` equivalents for positioned reads)
- rust-lang/rust#143725 (core: add Peekable::next_if_map)
- rust-lang/rust#145209 (Stabilize `path_add_extension`)
- rust-lang/rust#145342 (fix drop scope for `super let` bindings within `if let`)
- rust-lang/rust#145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- rust-lang/rust#145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- rust-lang/rust#145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- rust-lang/rust#145962 (Ensure we emit an allocator shim when only some crate types need one)
- rust-lang/rust#145963 (Add LSX accelerated implementation for source file analysis)
- rust-lang/rust#146054 (add `#[must_use]` to `array::repeat`)
- rust-lang/rust#146090 (Derive `PartialEq` for `InvisibleOrigin`)
- rust-lang/rust#146112 (don't uppercase error messages)
- rust-lang/rust#146120 (Correct typo in `rustc_errors` comment)
- rust-lang/rust#146124 (Test `rustc-dev` in `distcheck`)
- rust-lang/rust#146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- rust-lang/rust#146131 (rustdoc-search: add test case for indexing every item type)
- rust-lang/rust#146134 (llvm: nvptx: Layout update to match LLVM)
- rust-lang/rust#146136 (docs(std): add missing closing code block fences in doc comments)
- rust-lang/rust#146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- rust-lang/rust#146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- rust-lang/rust#146150 (fix(rustdoc): match rustc `--emit` precedence )
- rust-lang/rust#146155 (Make bootstrap self test parallel)
- rust-lang/rust#146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- rust-lang/rust#146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
github-actionsBot pushed a commit to model-checking/verify-rust-std that referenced this pull request Sep 9, 2025
@bors
Auto merge ofrust-lang#146185- Zalathar:rollup-n14lyv6, r=Zalathar
31e8eee
Rollup of 24 pull requests
Successful merges:
- rust-lang#140459 (Add `read_buf` equivalents for positioned reads)
- rust-lang#143725 (core: add Peekable::next_if_map)
- rust-lang#145209 (Stabilize `path_add_extension`)
- rust-lang#145342 (fix drop scope for `super let` bindings within `if let`)
- rust-lang#145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- rust-lang#145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- rust-lang#145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- rust-lang#145962 (Ensure we emit an allocator shim when only some crate types need one)
- rust-lang#145963 (Add LSX accelerated implementation for source file analysis)
- rust-lang#146054 (add `#[must_use]` to `array::repeat`)
- rust-lang#146090 (Derive `PartialEq` for `InvisibleOrigin`)
- rust-lang#146112 (don't uppercase error messages)
- rust-lang#146120 (Correct typo in `rustc_errors` comment)
- rust-lang#146124 (Test `rustc-dev` in `distcheck`)
- rust-lang#146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- rust-lang#146131 (rustdoc-search: add test case for indexing every item type)
- rust-lang#146134 (llvm: nvptx: Layout update to match LLVM)
- rust-lang#146136 (docs(std): add missing closing code block fences in doc comments)
- rust-lang#146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- rust-lang#146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- rust-lang#146150 (fix(rustdoc): match rustc `--emit` precedence )
- rust-lang#146155 (Make bootstrap self test parallel)
- rust-lang#146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- rust-lang#146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
Kobzol pushed a commit to Kobzol/portable-simd that referenced this pull request Feb 3, 2026
@bors
Auto merge of #146185 - Zalathar:rollup-n14lyv6, r=Zalathar
9b4e963
Rollup of 24 pull requests
Successful merges:
- rust-lang/rust#140459 (Add `read_buf` equivalents for positioned reads)
- rust-lang/rust#143725 (core: add Peekable::next_if_map)
- rust-lang/rust#145209 (Stabilize `path_add_extension`)
- rust-lang/rust#145342 (fix drop scope for `super let` bindings within `if let`)
- rust-lang/rust#145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- rust-lang/rust#145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- rust-lang/rust#145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- rust-lang/rust#145962 (Ensure we emit an allocator shim when only some crate types need one)
- rust-lang/rust#145963 (Add LSX accelerated implementation for source file analysis)
- rust-lang/rust#146054 (add `#[must_use]` to `array::repeat`)
- rust-lang/rust#146090 (Derive `PartialEq` for `InvisibleOrigin`)
- rust-lang/rust#146112 (don't uppercase error messages)
- rust-lang/rust#146120 (Correct typo in `rustc_errors` comment)
- rust-lang/rust#146124 (Test `rustc-dev` in `distcheck`)
- rust-lang/rust#146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- rust-lang/rust#146131 (rustdoc-search: add test case for indexing every item type)
- rust-lang/rust#146134 (llvm: nvptx: Layout update to match LLVM)
- rust-lang/rust#146136 (docs(std): add missing closing code block fences in doc comments)
- rust-lang/rust#146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- rust-lang/rust#146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- rust-lang/rust#146150 (fix(rustdoc): match rustc `--emit` precedence )
- rust-lang/rust#146155 (Make bootstrap self test parallel)
- rust-lang/rust#146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- rust-lang/rust#146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
Kobzol pushed a commit to Kobzol/portable-simd that referenced this pull request Feb 3, 2026
@bors
Auto merge of #146185 - Zalathar:rollup-n14lyv6, r=Zalathar
c483c86
Rollup of 24 pull requests
Successful merges:
- rust-lang/rust#140459 (Add `read_buf` equivalents for positioned reads)
- rust-lang/rust#143725 (core: add Peekable::next_if_map)
- rust-lang/rust#145209 (Stabilize `path_add_extension`)
- rust-lang/rust#145342 (fix drop scope for `super let` bindings within `if let`)
- rust-lang/rust#145750 (raw_vec.rs: Remove superfluous fn alloc_guard)
- rust-lang/rust#145827 (On unused binding or binding not present in all patterns, suggest potential typo of unit struct/variant or const)
- rust-lang/rust#145932 (Allow `inline(always)` with a target feature behind a unstable feature `target_feature_inline_always`.)
- rust-lang/rust#145962 (Ensure we emit an allocator shim when only some crate types need one)
- rust-lang/rust#145963 (Add LSX accelerated implementation for source file analysis)
- rust-lang/rust#146054 (add `#[must_use]` to `array::repeat`)
- rust-lang/rust#146090 (Derive `PartialEq` for `InvisibleOrigin`)
- rust-lang/rust#146112 (don't uppercase error messages)
- rust-lang/rust#146120 (Correct typo in `rustc_errors` comment)
- rust-lang/rust#146124 (Test `rustc-dev` in `distcheck`)
- rust-lang/rust#146127 (Rename `ToolRustc` to `ToolRustcPrivate`)
- rust-lang/rust#146131 (rustdoc-search: add test case for indexing every item type)
- rust-lang/rust#146134 (llvm: nvptx: Layout update to match LLVM)
- rust-lang/rust#146136 (docs(std): add missing closing code block fences in doc comments)
- rust-lang/rust#146137 (Disallow frontmatter in `--cfg` and `--check-cfg` arguments)
- rust-lang/rust#146140 (compiletest: cygwin follows windows in using PATH for dynamic libraries)
- rust-lang/rust#146150 (fix(rustdoc): match rustc `--emit` precedence )
- rust-lang/rust#146155 (Make bootstrap self test parallel)
- rust-lang/rust#146161 ([rustdoc] Uncomment code to add scraped rustdoc examples in loaded paths)
- rust-lang/rust#146172 (triagebot: configure some pings when certain attributes are used)
r? `@ghost`
`@rustbot` modify labels: rollup
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KobzolKobzolKobzol left review comments
+1 more reviewer
@klensyklensyklensy left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@KobzolKobzol

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcS-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Projects

None yet

Milestone

1.91.0

Development

Successfully merging this pull request may close these issues.

5 participants

@bit-aloo@Kobzol@bors@klensy@rustbot