Make re-export collection deterministic

[Aaron1011]

Fixes#65036

Previously, we were using an FxHashMap to collect module re-exports.
However, re-exports end up getting serialized into crate metadata, which
means that metadata generation was non-deterministic. This resulted in
spurious error messages changes (e.g. PR #64906) due to pretty-printing
implicitly depending on the order of re-exports when computing the
proper path to show to the user.

See #65042 for a long-term strategy to detect this kind of issue

[rust-highfive]

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

[Aaron1011]

r? @Centril

[Centril]

Looks good but r? @petrochenkov should really review this since resolve is their wheelhouse. :)

[Mark-Simulacrum]

I think we usually deal with this via collection into a Vec and sorting just before metadata serialization; that might be a better approach here?

[jonas-schievink]

Fixes #65043

Did you mean "Fixes #65036"?

[Centril]

I personally feel that IndexMap is a more reliable by-construction approach that takes care of every iteration site so you don't have to care about all of them. Maybe when we get the wrapper for FxHashMap we should use that instead.

[Aaron1011]

@jonas-schievink: Thanks, fixed

[Mark-Simulacrum]

We do have a StableMap in the data structures crate as of a recent PR, so maybe using that here would be better. It does not expose iteration order at all (lacking those methods entirely).

But again, I'm fine landing this in the mean time. An indexmap seems like a reasonable answer too.

[matthewjasper]

Stable map looks unsuitable since Idents don't implement Ord.

[petrochenkov]

I think we usually deal with this via collection into a Vec and sorting just before metadata serialization

That was my first reaction too.
Changing Resolutions, which are used for a thousand other things beside serializing reexports, is such an unproportionally heavy hammer for fixing this.

We actually have a method for visiting resolutions in a stable fashion (fn for_each_child_stable) which is already used in diagnostic reporting with the same purpose.

[petrochenkov]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 2c9bf5f9320945e4fc29927e8de60dd2fad8172f with merge 9f13bf7d334cc30a5fbe83d0309ccf107c0d512e...

[rust-timer]

Awaiting bors try build completion

[bors]

⌛ Trying commit 2c9bf5f9320945e4fc29927e8de60dd2fad8172f with merge 0e298d5daf4f422edf242c37161329ee79515942...

[petrochenkov]

I personally feel that IndexMap is a more reliable by-construction approach

Actually, I'm not sure that the insertion order itself is stable for the resolutions, if it's not, then IndexMap maintaining it won't matter much.

[Centril]

Interesting. I'm pretty happy I reassigned this PR. :)

[bors]

☀️ Try build successful - checks-azure
Build commit: 0e298d5daf4f422edf242c37161329ee79515942 (0e298d5daf4f422edf242c37161329ee79515942)

[rust-timer]

Queued 0e298d5daf4f422edf242c37161329ee79515942 with parent 032a53a, future comparison URL.

[bjorn3]

I personally feel that IndexMap is a more reliable by-construction approach

Actually, I'm not sure that the insertion order itself is stable for the resolutions, if it's not, then IndexMap maintaining it won't matter much.

We have StableMap since #64131. It works the same as FxHashMap, but you can't iterate it, you have to use .into_sorted_vec() instead.

[Aaron1011]

@petrochenkov: This looks like a wash performance wise - a few minor speedups and a few minor slowdowns. I suspect that much of it is just noise.

[petrochenkov]

Yeah, I just wanted to check what performance impact IndexMap can have, sorry for all this noise.
I still think using for_each_child_stable in finalize_resolutions_in is a preferable solution.

[Aaron1011]

@petrochenkov: All other things being equal, I think it's better to reduce the amount of non-determinism in the compiler. This issue was a pain to track down, and I'm worried that it might crop up again if we're relying on every caller using for_each_child_stable whenever they need to iterate over to it.

[petrochenkov]

Let's check whether IndexMap provides what we need or not - #65117.

[petrochenkov]

Looks like resolutions are added in stable order, so we can use an IndexMap.

@Aaron1011
Could you remove for_each_child_stable in this PR then, and replace its uses with for_each_child.
(Replacing other iterations through resolutions would also be nice, the use of for_each_child looks a bit cleaner in all cases.)

[Aaron1011]

@petrochenkov Updated

[petrochenkov]

Thanks!
@bors r+

[bors]

📌 Commit add0a42 has been approved by petrochenkov

[bors]

⌛ Testing commit add0a42 with merge 0358617...

[bors]

☀️ Test successful - checks-azure
Approved by: petrochenkov
Pushing 0358617 to master...

[RalfJung]

Previously, we were using an FxHashMap to collect module re-exports.
However, re-exports end up getting serialized into crate metadata, which
means that metadata generation was non-deterministic.

Clarification to avoid spreading this myth further: FxHashMapis deterministic. However, inserting a new element can surprisingly affect the entire iteration order. Also see #63713 (comment).

[Aaron1011]

@RalfJung: I meant 'non-deterministic with respect to platforms' - that is, the crate metadata order wouldn't look the same across different platforms. Sorry for the confusion.