Overhaul MSVC linker and Windows SDK detection code

[retep998]

What I've done here is try to make the code match what vcvars does much more closely. It now chooses which SDK to find based on the version of MSVC that it found. It also bases the decision of whether to find all the things on whether VCINSTALLDIR has been set, which is more likely to have only been set by an invocation of vcvars, unlike previously where it would do some things only if LIB wasn't set even though there was a valid use case for libraries to add themselves to LIB without having invoked vcvars.

There are still some debug println!s so people can test the PR and make sure it works correctly on various setups.

It supports VS 2015, 2013, and 2012. People who want to use versions of VS older (or newer) than that will have to manually invoke the appropriate vcvars bat file to set the proper environment variables themselves.

Do not merge yet.

Fixes#30229

[retep998]

r? @alexcrichton

[rust-highfive]

r? @Aatch

(rust_highfive has picked a reviewer for you, use r? to override)

[retep998]

cc @vadimcn

[retep998]

Testing by @DanielKeep confirms that this also works with the standalone Microsoft Visual C++ Build Tools 2015 Technical Preview, providing an option for users that want to use msvc but without installing the entire VS IDE.

[alexcrichton]

Instead of reverse here you should also be able to do dirs.into_iter().rev().filter(...) below

[alexcrichton]

I'm a little worried about this logic for a few reasons:

• This is basically an entirely new implementation, there's essentially no connection to what Clang does. This is generally a red flag, but I also never found a reason for why Clang does what it does, so it may not be that bad.
• It looks like the way this is written will force us to update the compiler each time a new SDK is released. That may not be able to ever avoid, however.
• It's very difficult to test this. I'd like to at least try to poke around a bit locally, but unfortunately my VM is offline.

I think that if this passes manual testing that we should land this, however, as fixing compatibility with the most recent VS is likely more important for now. It'd be great, however, to get a definitive answer on what this logic should be (or perhaps providing a better guarantee that it'll work), but I unfortunately don't know how to do that :(.

cc @vadimcn

[vadimcn]

Is this because of env vars, or just to avoid having to detect bitness of the host OS?

[retep998]

Mainly just to avoid having to detect bitness of the host OS. I'll add a FIXME comment saying this should probably detect the host OS bitness.

[vadimcn]

I don't know which approach is better off the top of my head.
@retep998, could expand a bit on why you've decided to re-write this logic from scratch? Not that there's anything wrong with the new code (as far as I can tell).

It looks like the way this is written will force us to update the compiler each time a new SDK is released

The VC's registry structure will likely stay the same for the next few releases, so we could try to extrapolate versions into the future and hope that it works.
Also, maybe it'd be worth to introduce environment variable overrides (e.g. RUST_MSVC_LINKER, RUST_MSVC_LIBS) to allow users to fix things up without having to rebuild the compiler from source?

[retep998]

Currently you can already override the entire system by just executing the proper vcvars bat file causing this code to back off and just trust everything is setup correctly and it'll work. Since we never know what sort of changes will occur in new versions of VS (such as VS 2015 adding the UCRT and the Windows 10 SDK both of which required new logic), I think it's better to not try to guess and just add support for it once someone can actually test it out.

As for testing this, I have VS 2015 2013 and 2012, and by commenting out some of the versions in that VS version array I could test how it handles all three versions and it works perfectly. It falls back to the 8.1 SDK for 2015 when there's no 10 SDK, it uses the 8.1 SDK for 2013 and it uses the 8 SDK for 2012. It still would be nice to get people testing it on other setups though as I've done all my testing on 64-bit Windows 10 only.

This code should work fine for new releases of the Win10 SDK as I've borrowed the exact logic from vcvars to find the newest Win10 SDK. Considering Microsoft said that Windows 10 is their final version of Windows, this means we may be in the clear with regards to that. New versions of VS however are still a possibility although the user can always just execute vcvars to setup the environment themselves.

The reason I rewrote this was because I felt like it wasn't robust enough. This implementation should very closely mirror what vcvars does and I'm more willing to trust what Microsoft does than what clang does, at least with regards to Microsoft's own software.

[ghost]

We can now also download the standalone / command-line Build Tools 2015, without even installing full VS2015, which installs compilers+SDKs+other utilities to compile C/C++ code on Windows.

Although the package itself is a Technical Preview, but I have successfully compiled with it on Windows 7 (fresh install) to Windows 10 and build Windows XP+ compatible libs. The CL compiler version is same as the one shipped with VS2015 Update 1.

It seems like this CLI-base is in preparation for Server 2016 and Nano Server feature called "containers". Also Windows team is working to provide "docker containers" (to bridge with their "native containers") GUI/CLI, where installing VS won't be a viable option: https://github.com/docker/docker/pulls/jhowardmsft.

[ghost]

what clang does,

I know at least detecting Windows 10 SDK (which is now at a new location the first time in a long time compared to its predecessors) hasn't done correctly by gyp, the build system favored by Chrome and node.js native modules projects. Initially gyp was developed because cmake was considered not meeting the requirement. Chrome guys are now moving away (or already) from their own gyp to another build system by Google called gn which seems to have fixed this SDK detection issue (but it is definitely in amateur hour to be fully trusted).

[retep998]

@vadimcn
Really the reason I decided to overhaul this code was because it broke on the Windows 10 SDK code which was last touched by me so I felt somewhat responsible and also I was rather annoyed because the last time I touched it was to fix it being broken on the Windows 10 SDK stuff. So, I thought to myself, I will do this properly this time. Thus I threw out the old code, dug deep into vcvarsqueryregistry.bat, and made this PR.

[alexcrichton]

I agree with @vadimcn that we should definitely have an escape hatch for future proofing, but I also agree with @retep998 that we should have the necessary detection in place that if you're in a VC shell then rustc won't be doing anything fancy. I also agree that mirroring what Microsoft does is likely better than Clang, and for now we claim we need VS 2013+ anyway so as long as we're compatible with those I'm happy.

Thanks again @retep998! r=me with the debugging prints and such removed

[alexcrichton]

Or better yet actually, feel free to just convert them all to logging

[retep998]

Switched it to use debug! logging and rebased while I was at it.

[alexcrichton]

@bors: r+ 915cb37

[bors]

⌛ Testing commit 915cb37 with merge ecae5f2...

[bors]

💔 Test failed - auto-mac-64-nopt-t

[alexcrichton]

@bors: retry

On Mon, Dec 14, 2015 at 7:53 PM, bors notifications@github.com wrote:

[image: 💔] Test failed - auto-mac-64-nopt-t
http://buildbot.rust-lang.org/builders/auto-mac-64-nopt-t/builds/7445

—
Reply to this email directly or view it on GitHub
#30233 (comment).

[bors]

⌛ Testing commit 915cb37 with merge 98ef4cc...

[bors]

💔 Test failed - auto-mac-64-nopt-t

[retep998]

I'm getting somewhat annoyed at that mac buildbot and its spurious failures.

[Manishearth]

@bors: retry

[bors]

⌛ Testing commit 915cb37 with merge 9e63cec...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-opt

[brson]

Requested for backport in #30229

[alexcrichton]

Accepted for a backport