Skip to content

Optimize crate resolution for large workspace - #159763

Merged
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
nikic:resolve-crate-opt
Aug 4, 2026
Merged

Optimize crate resolution for large workspace#159763
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
nikic:resolve-crate-opt

Conversation

@nikic

@nikicnikic commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

By keeping a map from hash to crate number.

@rustbotrustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 23, 2026
@nikic

nikic commented Jul 23, 2026

Copy link
Copy Markdown
ContributorAuthor

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rust-bors

This comment has been minimized.

Sign in to view
@rustbotrustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 23, 2026
rust-borsBot pushed a commit that referenced this pull request Jul 23, 2026
@bors
Auto merge of#159763- nikic:resolve-crate-opt, r=<try>
dd1f563
Optimize crate resolution for large workspace
@rust-bors

rust-borsBot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

☀️ Try build successful (CI)
Build commit: dd1f563 (dd1f5634c7540ca852bd1c09e61c8b203a8e5799)
Base parent: 390279b (390279b302ca98ae270f434100ae3730531d1246)

@rust-timer

This comment has been minimized.

Sign in to view
@nikicnikic mentioned this pull request Jul 23, 2026
Merged
9 tasks
@rust-timer

rust-timer commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (dd1f563): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-7.6%[-15.9%, -1.7%]10
All ❌✅ (primary)--0

Max RSS (memory usage)

Results (primary 2.7%, secondary 4.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

meanrangecount
Regressions ❌
(primary)		2.7%[2.7%, 2.7%]1
Regressions ❌
(secondary)		4.2%[2.7%, 5.4%]3
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		--0
All ❌✅ (primary)2.7%[2.7%, 2.7%]1

Cycles

Results (secondary -1.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		10.9%[3.0%, 25.7%]3
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-5.2%[-9.1%, -3.0%]10
All ❌✅ (primary)--0

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 486.724s -> 486.28s (-0.09%)
Artifact size: 387.61 MiB -> 387.61 MiB (0.00%)

@rustbotrustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 23, 2026
@nikic
nikic marked this pull request as ready for review July 23, 2026 13:16
@rustbotrustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 23, 2026
@rustbotrustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jul 23, 2026
@rustbot

rustbot commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: compiler
  • compiler expanded to 74 candidates
  • Random selection from 20 candidates

bjorn3
bjorn3 reviewed Jul 23, 2026
View reviewed changes
Comment threadcompiler/rustc_metadata/src/creader.rs Outdated
@nikic
Optimize crate resolution
54ff9a2
By keeping a map from hash to crate number.
@nikic
nikicforce-pushed the resolve-crate-opt branch from 89418e2 to 54ff9a2CompareJuly 23, 2026 14:17
@bjorn3

bjorn3 commented Jul 23, 2026

Copy link
Copy Markdown
Member

Great find by the way!

@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

@rustbot reroll

@rustbotrustbot assigned khyperia and unassigned davidtwcoAug 4, 2026
@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

Or maybe @bjorn3 can you r+ it, since you looked at it, to avoid review ping pong? :)

@bjorn3

bjorn3 commented Aug 4, 2026

Copy link
Copy Markdown
Member

r? bjorn3

@bors r+

@rust-bors

rust-borsBot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit 54ff9a2 has been approved by bjorn3

It is now in the queue for this repository.

🌲 The tree is currently closed for pull requests below priority 10. This pull request will be tested once the tree is reopened.

Reason for tree closure: manually handling queue due to backlog

@rust-borsrust-borsBot added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Aug 4, 2026
@rust-borsrust-borsBot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 4, 2026
@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

Marking as rollup=iffy, because this only affects one benchmark, and we have a lot of rollup=never PRs in the queue.

@bors rollup=iffy

@JonathanBrouwerJonathanBrouwer mentioned this pull request Aug 4, 2026
Merged
rust-borsBot pushed a commit that referenced this pull request Aug 4, 2026
@bors
Auto merge of#160506- JonathanBrouwer:rollup-9ScDW58, r=JonathanBro…
1ed2df6
…uwer
Rollup of 10 perf-sensitive pull requests
Successful merges:
- #157281 (perf: skip irrelevant foreign impls when building the specialization graph)
- #159403 (Next steps for FnDef binder changes (instantiate most FnDef binders))
- #159763 (Optimize crate resolution for large workspace)
- #160033 (Speed up `EverInitializedPlaces`)
- #160268 (perf: store the fulfillment engine inline in ObligationCtxt)
- #160317 (perf: Cache already-checked types in the privacy visitor)
- #160399 (interpret: skip deref-projection validity checks when they are not needed)
- #160451 (Deduplicate target and host filesearch)
- #160453 (Add fast path to `escape_string_symbol`)
- #160454 (Add offload guard flags to typeck to prevent perf regressions)
@rust-bors
rust-borsBot merged commit adf7af8 into rust-lang:mainAug 4, 2026
13 checks passed
@rustbotrustbot added this to the 1.99.0 milestone Aug 4, 2026
rust-timer added a commit that referenced this pull request Aug 4, 2026
@rust-timer
Unrolled build for#159763
2f596f7
Rollup merge of #159763 - nikic:resolve-crate-opt, r=bjorn3
Optimize crate resolution for large workspace
By keeping a map from hash to crate number.
@JonathanBrouwer

JonathanBrouwer commented Aug 5, 2026

Copy link
Copy Markdown
Member

Verifying that actual perf results after merge match expected results
@rust-timer build 2f596f7

@rust-timer

This comment has been minimized.

Sign in to view
WhySoBad pushed a commit to WhySoBad/miri that referenced this pull request Aug 5, 2026
@bors
Auto merge of #160506 - JonathanBrouwer:rollup-9ScDW58, r=JonathanBro…
0a83571
…uwer
Rollup of 10 perf-sensitive pull requests
Successful merges:
- rust-lang/rust#157281 (perf: skip irrelevant foreign impls when building the specialization graph)
- rust-lang/rust#159403 (Next steps for FnDef binder changes (instantiate most FnDef binders))
- rust-lang/rust#159763 (Optimize crate resolution for large workspace)
- rust-lang/rust#160033 (Speed up `EverInitializedPlaces`)
- rust-lang/rust#160268 (perf: store the fulfillment engine inline in ObligationCtxt)
- rust-lang/rust#160317 (perf: Cache already-checked types in the privacy visitor)
- rust-lang/rust#160399 (interpret: skip deref-projection validity checks when they are not needed)
- rust-lang/rust#160451 (Deduplicate target and host filesearch)
- rust-lang/rust#160453 (Add fast path to `escape_string_symbol`)
- rust-lang/rust#160454 (Add offload guard flags to typeck to prevent perf regressions)
@rust-timer

rust-timer commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (2f596f7): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		--0
Improvements ✅
(secondary)		-7.5%[-15.3%, -1.7%]11
All ❌✅ (primary)--0

Max RSS (memory usage)

Results (primary -0.1%, secondary -0.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

meanrangecount
Regressions ❌
(primary)		1.0%[0.4%, 1.6%]2
Regressions ❌
(secondary)		0.7%[0.7%, 0.7%]1
Improvements ✅
(primary)		-2.3%[-2.3%, -2.3%]1
Improvements ✅
(secondary)		-1.0%[-1.2%, -0.7%]2
All ❌✅ (primary)-0.1%[-2.3%, 1.6%]3

Cycles

Results (primary 0.4%, secondary -2.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

meanrangecount
Regressions ❌
(primary)		0.9%[0.5%, 1.6%]7
Regressions ❌
(secondary)		1.0%[0.7%, 1.6%]4
Improvements ✅
(primary)		-1.2%[-1.3%, -1.1%]2
Improvements ✅
(secondary)		-4.1%[-10.2%, -0.7%]12
All ❌✅ (primary)0.4%[-1.3%, 1.6%]9

Binary size

Results (primary -0.0%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

meanrangecount
Regressions ❌
(primary)		--0
Regressions ❌
(secondary)		--0
Improvements ✅
(primary)		-0.0%[-0.1%, -0.0%]8
Improvements ✅
(secondary)		-0.1%[-0.1%, -0.0%]35
All ❌✅ (primary)-0.0%[-0.1%, -0.0%]8

Bootstrap: 489.577s -> 488.839s (-0.15%)
Artifact size: 390.24 MiB -> 390.29 MiB (0.01%)

flip1995 pushed a commit to flip1995/rust-clippy that referenced this pull request Aug 17, 2026
@bors
Auto merge of #160506 - JonathanBrouwer:rollup-9ScDW58, r=JonathanBro…
7554020
…uwer
Rollup of 10 perf-sensitive pull requests
Successful merges:
- rust-lang/rust#157281 (perf: skip irrelevant foreign impls when building the specialization graph)
- rust-lang/rust#159403 (Next steps for FnDef binder changes (instantiate most FnDef binders))
- rust-lang/rust#159763 (Optimize crate resolution for large workspace)
- rust-lang/rust#160033 (Speed up `EverInitializedPlaces`)
- rust-lang/rust#160268 (perf: store the fulfillment engine inline in ObligationCtxt)
- rust-lang/rust#160317 (perf: Cache already-checked types in the privacy visitor)
- rust-lang/rust#160399 (interpret: skip deref-projection validity checks when they are not needed)
- rust-lang/rust#160451 (Deduplicate target and host filesearch)
- rust-lang/rust#160453 (Add fast path to `escape_string_symbol`)
- rust-lang/rust#160454 (Add offload guard flags to typeck to prevent perf regressions)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bjorn3bjorn3bjorn3 left review comments

Assignees

@bjorn3bjorn3

Labels

S-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

1.99.0

Development

Successfully merging this pull request may close these issues.

8 participants

@nikic@rust-timer@rustbot@bjorn3@Kobzol@JonathanBrouwer@khyperia@davidtwco