Shrink the rust-src component

[cuviper]

Before this change, the installable rust-src component had essentially the same contents as the rustc-src dist tarball, just additionally wrapped in a rust-installer. As discussed on internals, rust-src is only meant to support uses for the standard library, so it doesn't really need the rest of the compiler sources.

Now rust-src only contains libstd and its path dependencies, which roughly matches the set of crates that have rust-analysis data. The result is significantly smaller, from 36MB to 1.3MB compressed, and from 247MB to 8.5MB uncompressed.

[rust-highfive]

r? @brson

(rust_highfive has picked a reviewer for you, use r? to override)

[malbarbo]

@cuviper What do you think about this rust-std component issue?

[cuviper]

@malbarbo To paraphrase, you're asking about the fact that rust-std may or may not include rustc crates? I'm not actually referring to rust-std for the chosen crates here, but only the proper std crate and its dependencies, including those that are arch-specific like the sanitizers.

If we want to include rustc crates in rust-src, we could, but I don't think that's necessary. All of those crates are unstable, and folks who really want that can still checkout rust.git for use by racer et al.

[alexcrichton]

Thanks for the PR @cuviper! This seems like the kind of list that's highly susceptible to breakage over time, I wonder if we could enhance distcheck to verify that the rust-src component is valid? Maybe something like executing cargo metadata or something like that in the un-tar'd version to verify that all dependencies are there at least?

Also cc @jonathandturner and @nrc, I believe the rls uses the rust-src component

[petrochenkov]

I need to check if this breaks intellisense for people developing rustc itself using editors with plugins depending on rust-src component (e.g. VSCode).

[petrochenkov]

I need to check if this breaks intellisense ...

Ok, everything still seems to work normally.

[cuviper]

@alexcrichton I agree that the list is a bit fragile. Using cargo metadata as a sanity check sounds like a good idea! Maybe I can even use that to generate the list in the first place, picking out the path+file:///... locations. I'll try it out.

I think rls primarily uses this through racer, but I'm not certain. I don't usually use rls or vscode, but it seems like it works, and I did make sure racer directly works too.

[cuviper]

Maybe I can even use that to generate the list in the first place, picking out the path+file:///... locations.

Well at first glance, it's not that easy, because cargo metadata in the full source gives me data on the entire workspace, including all the rustc bits. I guess that would have to be more clever and actually walk the dependencies in the metadata starting from std.

Just having the distcheck as your suggested is probably an 80% solution, and it's not like this list changes frequently.

[alexcrichton]

So far this list has changed at least once a cycle for I think the past 3 or 4 cycles? Which means it'd otherwise have broken once per release so far for the past few months :(

[cuviper]

OK, there's a distcheck. I double-checked that if the source list has something bogus then rust-src doesn't build successfully, and if it's missing something (say libcore) then the distcheck catches that.

[alexcrichton]

@bors: r+

Awesome, thanks @cuviper!

[bors]

📌 Commit b9bdb17 has been approved by alexcrichton

[cuviper]

Hmm, it looks like this was partially merged in rollup #41567, but without the distcheck commit. Then rollup #41585 says it's including this too, but doesn't have any of my commits. Is @bors confused?

[alexcrichton]

@frewsxcv did you happen to manually back this out of #41585?

@cuviper thankfully the way we do rollups means that a PR is only closed if it merges all commits! In that sense I'm hoping that if @bors naturally gets around to this PR it'll merge and end up picking up the last commit.

[frewsxcv]

I did not do any manual adjustments to the rollup. Everything that's committed in there is through the bors queue interface. I feel like we've had this happen before; it's kind of a scary bug.

[cuviper]

📌 Commit b9bdb17 has been approved by alexcrichton

I just noticed, that's not the tail commit -- it should be c5cd4cb. @alexcrichton maybe r+ it explicitly?

[alexcrichton]

@bors: r+ c5cd4cb

[bors]

💡 This pull request was already approved, no need to approve it again.

[bors]

🙀 c5cd4cb51e1a77cd4f10ce2d04fc29621354afc9 is not a valid commit SHA. Please try again with b9bdb17.

[alexcrichton]

Hm sure enough homu is now wedged with this, closing to test out.

[alexcrichton]

Hm @cuviper mind sending up a follow-up PR? Sorry for the automation hiccup :(

[cuviper]

No problem, I was already getting that ready. :) #41608