Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T].

[MaulingMonkey]

No idea if these changes are reasonable - please feel free to suggest changes/rewrites. And these are some of my first real commits to any rust codebase - don't be gentle, and nitpick away, I need to learn! ;)

Overview

Embedding .natvis files into .pdbs allows MSVC (and potentially other debuggers) to automatically pick up the visualizers without having to do any additional configuration (other than to perhaps add the relevant .pdb paths to symbol search paths.)

The native debug engine for MSVC parses the type names, making various C++ish assumptions about what they mean and adding various limitations to valid type names. &str cannot be matched against a visualizer, but if we emit str& instead, it'll be recognized as a reference to a str, solving the problem. [T] is similarly problematic, but emitting slice<T> instead works fine as it looks like a template. I've been unable to get e.g. slice<u32>& to match visualizers in VS2015u3, so I've gone with str* and slice<u32>* instead.

Possible Issues

• I'm not sure if slice<T> is a great mangling for [T] or if I should worry about name collisions.
• I'm not sure if linker.rs is the right place to be enumerating natvis files.
• I'm not sure if these type name mangling changes should actually be MSVC specific. I recall seeing gdb visualizer tests that might be broken if made more general? I'm hesitant to mess with them without a gdb install. But perhaps I'm just wracking up technical debt.
  Should I try pacman -S mingw-w64-x86_64-gdb and to make things consistent?
• I haven't touched const / mut yet, and I'm worried MSVC might trip up on mut or their placement.
• I may like terse oneliners too much.
• I don't know if there's broader implications for messing with debug type names here.
• I may have been mistaken about bellow test failures being ignorable / unrelated to this changelist.

Test Failures on x86_64-pc-windows-gnu

---- [debuginfo-gdb] debuginfo-gdb\associated-types.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\associated-types.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16
note: Run with `RUST_BACKTRACE=1` for a backtrace.
[...identical panic causes omitted...]
---- [debuginfo-gdb] debuginfo-gdb\vec.rs stdout ----
thread '[debuginfo-gdb] debuginfo-gdb\vec.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16

Relevant Issues

• Metaissue for Visual Studio debugging Rust #40460 Metaissue for Visual Studio debugging Rust
• Investigate natvis for improved msvc debugging #36503 Investigate natvis for improved msvc debugging
• Debug visualization of Rust data structures PistonDevelopers/VisualRust#160 Debug visualization of Rust data structures

Pretty Pictures

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @eddyb (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[shepmaster]

Thanks so much for the PR @MaulingMonkey! I can't speak to the content, but I must say that that is a top-notch commit message! We'll get someone knowledgable to review this shortly!

[Mark-Simulacrum]

r? @brson I think

[arielb1]

r? @michaelwoerister

[michaelwoerister]

Thanks a lot for the PR, @MaulingMonkey! This looks like a great contribution!

• I'm not sure if slice is a great mangling for [T] or if I should worry about name collisions.

slice<T> does seem a little problematic but it should actually be fine. User-defined types normally start with an uppercase letter so we can leave it for now, I think.

• I'm not sure if linker.rs is the right place to be enumerating natvis files.

At a first glance, this looks like the right place.

• I'm not sure if these type name mangling changes should actually be MSVC specific. I recall seeing gdb visualizer tests that might be broken if made more general? I'm hesitant to mess with them without a gdb install. But perhaps I'm just wracking up technical debt.

Staying MSVC specific for this is the way to go, I'd say.

• I haven't touched const / mut yet, and I'm worried MSVC might trip up on mut or their placement.

You can worry about that in another PR :)

• I may like terse oneliners too much.

Yeah, they are in conflict with rustc's current code style guide.

• I don't know if there's broader implications for messing with debug type names here.

As long as this is limited to MSVC and that still works as good or better than before, this looks good to me.

• I may have been mistaken about bellow test failures being ignorable / unrelated to this changelist.

There are no tests for MSVC debugging yet. There might be a handful of other tests that depend debuginfo type names, but CI will tell us about those soon enough.

Now for a question of mine: Is there a way to transform the mangled name back to something more Rust-like when they are displayed?

[MaulingMonkey]

Thanks for the kind words all!

• I may like terse oneliners too much.

Yeah, they are in conflict with rustc's current code style guide.

Okay, I've got a followup changelist fixing this, that I've got the build and test scripts chewing through. Any other style issues while I'm in there?

I'm also inclined to rename, comment on, and move this to the top of push_debuginfo_type_name, to better describe "what" and "why":

// When targeting MSVC, emit C++ style type names for compatability with// .natvis visualizers (and perhaps other existing native debuggers?)let cpp_like_names = cx.sess().target.target.options.is_like_msvc;

As long as this is limited to MSVC and that still works as good or better than before, this looks good to me.

The only regression I can think of is if someone built with MSVC but then wanted to debug with GDB instead for some reason (debugging binaries that someone else built?)

Now for a question of mine: Is there a way to transform the mangled name back to something more Rust-like when they are displayed?

Not that I'm aware of, short of writing a full blown debug engine (e.g. maybe on VisualRust's far future roadmap?) Such a .natvis feature would be useful to unexpand C++ templates too, if it were an option (e.g. std::basic_string<char, ...> -> std::string)

[retep998]

The only regression I can think of is if someone built with MSVC but then wanted to debug with GDB instead for some reason

This is already incredibly unsupported and doesn't work due to vastly different debuginfo formats. I wouldn't be worried about this.

[michaelwoerister]

Command::arg() should be able to take a OsStr. You probably can't use format! with that but you could build up the OsString with its push method.

[MaulingMonkey]

Good catch! Testing a patch now.

[michaelwoerister]

Any other style issues while I'm in there?

Didn't see any.

If you support non-unicode paths too (see comment) then I'd be happy to r+ this.

[MaulingMonkey]

If you support non-unicode paths too (see comment) then I'd be happy to r+ this.

Good catch - using traits for "overloading" was new to me, so I'd mistakenly assumed Command::arg() took &str. Latest changelist fixes this.

[Boddlnagg]

@MaulingMonkey: Have you tested the string visualizers with non-ASCII strings? Since Rust strings are defined to be UTF-8, we should use the s8 modifier in the natvis file, as in:

<TypeName="str">
<DisplayString>{data_ptr,[length]s8}</DisplayString>
<StringView>data_ptr,[length]s8</StringView>
...

I found out about this some time ago, but the official documentation is sparse unfortunately. It's listed on https://msdn.microsoft.com/en-us/library/75w45ekt.aspx, but it doesn't say that it means UTF-8. It does say, however, that s32 means UTF-32.
NatVis seems to be not very well documented in general, the \NATVIS linker switch is also basically undocumented ...

[MaulingMonkey]

@MaulingMonkey: Have you tested the string visualizers with non-ASCII strings?

I have now. With your suggested changes, I grabbed https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt . Breakpointing slice2 of:

use std::io::BufReader;use std::io::BufRead;use std::fs::File;use std::path::Path;fnmain(){let f = File::open("I:\\home\\projects\\test\\rust\\UTF-8-demo.txt").unwrap();letmut file = BufReader::new(&f);for line in file.lines(){let line = line.unwrap();let slice :&str = &line;let slice2 = slice;println!("{}", slice);}}

And stepping to this line:

 ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β), ⎪⎢⎜│───── ⎟⎥⎪

Shows me the following with the s8 postfix, which looks accurate. The enums could perhaps use some improvement, and there's a few placeholder boxes for characters my visualizer font is missing, but it's much better than what was spewed out without the s8 postfixes for these lines.

The combining mark for the second T in " STARGΛ̊TE" is also misplaced to the left of the T in the Value column, but not in the full blown Text Visualizer. Then again, Chrome also shows it misplaced to the left of the T in this Chrome tab, but not in the original text file location, so this might just be more font issues?

I can roll those changes into this pull request.

[MaulingMonkey]

One caveat is that the expanded array representation is still individual ASCII-ish characters - ArrayItems/ValuePointer will just plain ignore the postfix if added - but that might be a feature anyways.

[Boddlnagg]

Just as an addendum, I found the link where s8 is documented a little bit more: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/dx--display-visualizer-variables-

[alexcrichton]

ping @michaelwoerister for a follow-up review, this looks ready to go!

[michaelwoerister]

@bors r+

[bors]

📌 Commit 90a7cac has been approved by michaelwoerister

[bors]

⌛ Testing commit 90a7cac with merge 6f815ca...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: michaelwoerister
Pushing 6f815ca to master...

[michaelwoerister]

Thanks again, @MaulingMonkey, for kicking this off!

[MaulingMonkey]

My pleasure. Thanks for the review, @michaelwoerister ! And thanks @Boddlnagg (and more!) for cluing me into the /NATVIS flag and type name issues. Your research turned this into low hanging fruit, convincing me I could help 😃

[retep998]

For the record, this breaks people trying to use LLD with msvc because LLD doesn't yet understand /NATVIS.

[Boddlnagg]

Does rustc know that LLD is being used? Then we should just disable /NATVIS in that case.

[retep998]

Currently I'm just passing -Clinker="C:\Program Files\LLVM\bin\lld-link.exe" to try out LLD. I'm not sure what the "correct" way to use LLD is.

[MaulingMonkey]

Possible temporary workarounds to unblock your testing:

• Temporarily move the .natvis files to another path where rustc won't find them.
• -Z linker-flavor=ld ?

I imagine -Clinker=... sets TargetOptions::linker, which can probably be checked as a more permanent workaround (e.g. skip natvis files if the linker contains "lld-link"?) Is LLD still missing debuginfo support per #39915 (comment) ?

Fixing this on LLD's end might be as simple as adding def natvis : QF<"natvis">; to https://github.com/llvm-mirror/lld/blob/master/COFF/Options.td#L113-L144 which might be preferred (e.g. if LLD does eventually add this feature, it'd be preferable not to have a workaround on rust's end hanging around in need of removal)

If I do take a stab at a workaround on rustc's end, would http://releases.llvm.org/5.0.0/LLVM-5.0.0-win64.exe work for testing, or have you installed LLVM via another method that I should prefer?

[MaulingMonkey]

A similar patch for /GUARDSYM: llvm-mirror/lld@96a6f2c .

Tempted to find someone who's already set up for committing to/testing LLVM - perhaps on one of their mailing lists if nobody here already is - a job has eaten my free time. Maybe the pcc fellow who implemented https://reviews.llvm.org/D37607 ?

[retep998]

@MaulingMonkey

I went with the former workaround. I don't know why anyone would use the latter workaround considering how horrifying ld is.

LLD does support emitting debuginfo now (http://blog.llvm.org/2017/08/llvm-on-windows-now-supports-pdb-debug.html). It's definitely a lot closer to the point of being a legitimate alternative.

I would prefer LLD supporting the flag or at least ignoring it and not erroring. But until then it would probably be better to not pass /NATVIS to lld for the time being.

That is a valid way to acquire LLVM, although I personally use http://llvm.org/builds/ to get more bleeding edge builds.

[MaulingMonkey]

I've created a potential workaround patch for rustc, although I can't yet test it as ./x.py build is generating millions of warnings (literally) building LLVM, which is taking awhile. EDIT: I've made it public here while waiting to build: MaulingMonkey@d953c5a - I'm half tempted to make an untested pull request and let the build servers sort it out, although that feels like bad juju.

I've also created a patch for lld-link which works, now I just need to hand it off for submission somehow. EDIT: I've sent this off to llvm-commits@lists.llvm.org as

[lld] Small patch to ignore "/natvis:..." flag in the COFF / lld-link.exe frontend.

although it's been held in moderation thanks to screwing up my email aliases.

[MaulingMonkey]

LLVM patch landed:
http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20170911/486071.html
llvm-mirror/lld@27b9c42

Still stuck in trying to resolve build hell for the rustc patch.

[michaelwoerister]

Awesome, thanks @MaulingMonkey!

[MaulingMonkey]

I've updated https://github.com/MaulingMonkey/rust/tree/lld-link-natvis-regression-fix which at least builds on x86_64-pc-windows-gnu. I suspect it probably works. I still cannot make a x86_64-pc-windows-msvc build to actually test on however - the last build I canceled accumulated 1.2GB of stdout logs over 24 hours and showed no signs of being anywhere close to done. I'll avoid canceling my current build, although I don't know how long it will take. This doesn't look like it's been resolved by my fresh clone either.

I'll submit a pull request once any of the following happen:

• My msvc build finishes, allowing me to test and verify the workaround works.
• Someone else manages to build, test, and verify the workaround works.
• Someone tells me testing isn't important

[retep998]

The massive amounts of logs are due to a gcc regression which was fixed and will be landing in Rust soon#44531

[MaulingMonkey]

@retep998 thanks for the link - I was able to merge that for testing before it landed in trunk. Work has eaten a lot of my free time, but I've managed to cobble together a pull request that actually works when tested, not just works in my head 😄