Run decorators on expanded AST

[jseyfried]

Fixes#32950.
r? @nrc

[jseyfried]

cc @durka

[nrc]

This is (I think) a breaking change since a decorator author could be using macros in the AST as input to their functionality.

I think I prefer the expansion order to be the same for all attributes, seems confusing to have difference expansion orders. Other than #32950, is there any reason to do this? re #32950, I think a better fix is to implement visit_mac and issue an error rather than panicking.

[durka]

What if the decorator produces more decorators?

[durka]

cc #33769, the previous attempt to fix this

[jseyfried]

@durka decorator-generated items would still get expanded.

[jseyfried]

@nrc

This is (I think) a breaking change

Agreed, but I'm pretty sure it wouldn't break anything in practice.

seems confusing to have difference expansion orders

I agree that this would be a downside.

Other than #32950, is there any reason to do this?

I believe some use cases of decorators might benefit from having access to expanded AST, but I don't have any specific examples in mind (besides #32950, of course).

In lieu of specific examples, a thought experiment: if we allowed macros in field declaration positions (not saying this would be a good idea), for example

structFoo{x:i32,
mac!(),//< say `mac!()` expands to `y: i32`}

then derive would need access to expanded AST.

[jseyfried]

That being said, I only weakly support this PR -- @nrc if you still don't think this is a good idea I'll close.

[durka]

Only a plugin-breaking-change, to be clear, since the only stable "decorator author" is rustc.

[durka]

It does seem to make some intuitive sense that a decorator on an item would be the last thing to be expanded, after any macros which generate parts of the item. But what about mac! { #[derive(Foo)] struct Item { stuff: thing!() } }, what is the order there?

[jseyfried]

@durka

Only a plugin-breaking-change, to be clear

Agreed, although it could theoretically break code that uses (unstable) decorator plugins without breaking the plugins themselves (highly unlikely, imo).

what about mac! { #[derive(Foo)] struct Item { stuff: thing!() } }, what is the order there?

We would expand mac! first. Before expanding mac!, #[derive(Foo)] is just a part of a token tree -- we don't know if it will end up being a decorator or if it has some other meaning, for example:

macro_rules! mac {(#[derive($i:ident)] $it:item) => { println!(stringify!($i));}}fnmain(){mac!{ #[derive(Foo)]structBar;}// prints "Foo"}

After expanding mac!, say we end up with #[derive(Foo)] struct Item { stuff: thing!() }. Then, we would expand expand Item (i.e. we would expand thing!) and finally decorate Item.

[nrc]

Hmm, I'm still not super-happy about having different expansion orders, but this seems to be a fine solution other than that. I think it should probably land. We'll have an opportunity to change direction with the procedural macro overhaul in any case.

@bors: r+

[bors]

📌 Commit 635a82e has been approved by nrc

[bors]

⌛ Testing commit 635a82e with merge ff13155...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-32cross-opt, auto-linux-64-cargotest, auto-linux-64-cross-armhf, auto-linux-64-cross-armsf, auto-linux-64-cross-freebsd, auto-linux-64-cross-netbsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-cross-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-mir, auto-win-msvc-64-opt-rustbuild