Set CompilerDesugaring expn_info for all desugared expressions

[sinkuu]

This PR adds CompilerDesugaring expn_info for all compiler desugared expressions. This will ease writing lints which need to exclude them (for example https://github.com/Manishearth/rust-clippy/pull/1513).

However, it will add in this macro invocation note for every error on syntax-sugar span, which is unhelpful. The 2nd commit removes them.

[sinkuu]

It looks like the highfive bot has missed this PR. Can anyone review this?

[bors]

☔ The latest upstream changes (presumably #40091) made this pull request unmergeable. Please resolve the merge conflicts.

[oli-obk]

r? @Manishearth

noone seems to want to take this, and it's clippy related ;)

[alexcrichton]

cc @rust-lang/compiler

[eddyb]

r? @jseyfried

[Manishearth]

I would avoid giving sole r+ on PRs that affect clippy

[nrc]

I think we should not do this - we have avoided it in the past. I have tried to keep a strong distinction in the compiler around generated code - in particular lowered code is not generated code. The contract is basically that if the user should recognise that the code is generated (i.e., they used a macro or other form of codegen) then the span should have an expn_info and error messages should be explicit about the expansion. Tools can recognise that the source code does not reflect the compiler's representation and can 'correct' that via the expansion stack.

OTOH, code that the user cannot recognise as generated (e.g., for loops) should not have an expn_info and error messages should be opaque about the lowering - i.e., they must behave as if the source code were first class. Spans should be adjusted by lowering in such a way that tools should be able to cope with just the spans and not need 'expansion' information.

I'm not sure how best to help Clippy here. I suspect it should be using a higher level API to the compiler's info, but not sure how that should work in general.

[eddyb]

@nrc For precise spans we need to do this though, however I agree users shouldn't be able to see it.

[nrc]

@eddyb yeah, long term I think we should consider changes - I do envision lowering being reflected in the spans, but distinct from other kinds of generated code (this might just be a flag, but ideally we'd have some way of handling the 'macro definition').

[eddyb]

@nrc FWIW we already create such spans - we have to, for allow_internal_unstable in some cases.

[nrc]

@eddyb I'm not sure what you mean by "such spans" - if we did this for lowered expressions then this PR would be redundant, no? I'm not sure how the stability checking works though.

[eddyb]

@nrc We only do it when have to for stability, look for calls to the allow_internal_unstable method.

[nikomatsakis]

@nrc@eddyb

Would it be helpful to have a "lowered code" distinction that applies specifically to lowering (and not things the user understands as generated)? That's what I would expect. (Caveat: I've not looked at PR yet.)

[nikomatsakis]

I definitely think we should not be writing this to the output.

[sinkuu]

These lines are required by compiletest, but not shown in rustc output (see the second commit).

[nikomatsakis]

Nor this.

[eddyb]

@nikomatsakis We already have that distinction, we use it for creating allow_internal_unstable spans during HIR lowering.

[nikomatsakis]

@eddyb so ... what are we arguing about exactly? I must be confused.

[eddyb]

@nikomatsakis I am saying we're already using this in some cases, and I'm pretty sure we agree we need unique spans in the future, so I'm in favor of making this change with no user-visible output.

[nikomatsakis]

@eddyb ok I agree with you :)

[nrc]

So I assume the allow_internal_unstable change had a negative impact on save-analysis and that doing this universally will break it further. It took a fair amount of tweaking to get the HIR lowering working with save-analysis and that change goes against that design. I would like to see save-analysis fixed before going further down this road (and it would be nice, in general, to get pinged about any changes to spans or macro expansion, rather than have them buried in a 4000 line PR and finding out 4 months later).

[eddyb]

I'm not sure how that makes sense? save-analysis doesn't look at HIR spans, does it?

[nrc]

save-analysis certainly looks at HIR nodes (we have to in order to get type check info). We also look at spans from deep inside nodes. I'm not sure if we actually look at spans from HIR nodes because of that, so you are right that it is probably not as bad as I thought. However, it is certainly possible that we do look at some HIR span somewhere, so if we move to doing the span stuff everywhere, then I'd like to make sure we don't break any save-analysis stuff.

[eddyb]

@nrc You should auto-generate a corpus of tests (from run-pass tests or something), IMO.
That way we get to know ahead of time if we're breaking save-analysis.

[nrc]

@eddyb yeah, tests for save-analysis have been on my mind for like the last 6 months. Someday I'll implement them

[bors]

☔ The latest upstream changes (presumably #39921) made this pull request unmergeable. Please resolve the merge conflicts.

[oli-obk]

So what action is required here?

[nrc]

@oli-obk@sinkuu I think this PR needs changing so that the changes are not user-visible (see @nikomatsakis's comments). There is apparently a distinction that can be made between lowered and generated code and that should be used (I was under the impression we couldn't directly do this, but perhaps I don't understand exactly what is meant here) - the stuff in this PR should use that.

Finally, we should establish that this does not break save-analysis. To do that, fine the two run-make save-analysis tests. Each should emit a JSON or CSV file when run. Compare the files before and after applying this PR, they should be identical. If the post-PR file is smaller, then there are issues that need addressing.

[oli-obk]

@sinkuu needs another rebase

@nrc: the issues have been addressed. The change is only user-visible in the json output.

[nrc]

@sinkuu did you get a chance to compare the save-analysis output?

[nrc]

The change is only user-visible in the json output.

I think desugarings should not be user-visible in JSON output either - the expectation is that JSON errors are shown to users fairly directly by tools. I wouldn't expect an IDE (for example) to show the user the macro expansion of a desugared language feature.

[carols10cents]

☔️ merge conflicts!

[arielb1]

@sinkuu & @nrc - what is the status of this PR?

[arielb1]

🕸 We haven't seen activity on this PR for 2 weeks 🕸, so we're closing it to reduce our backlog. Feel free to reopen if you still want to work on it.