Extended expand.rs to support alternate expansion behaviours (eg. stepwise expansion)

[DanielJCampbell]

r? nrc

[nrc]

r? @nrc

[nrc]

Could you explain precisely when the closure is called.

[DanielJCampbell]

Updated with the suggested changes - definitely looks tidier with the macros gone.

[nrc]

@bors: r+

[bors]

📌 Commit 866da14 has been approved by nrc

[bors]

⌛ Testing commit 866da14 with merge 68303be...

[alexcrichton]

@bors: retry force clean

• restarted buildbot

[jseyfried]

@DanielJCampbell
Could you elaborate on what you mean by "stepwise expansion"? What is the intended use case? In particular, is this intended to support a plugin or is it groundwork for future changes to the compiler?

I ask because I'm not sure how this will interact with macro modularization, which we're planning on prototyping in the next couple of weeks.

[DanielJCampbell]

@jseyfried, @nrc
By stepwise I mean if you change the closures to return identity (|_, node| node|) macros only expand one step - you can feed in a crate repeatedly, and with some bookkeeping end up with a series of crates at each successive level of expansion. This is intended to support a plugin I hope to add to save analysis in the next week or two.

[nrc]

@bors: r-

[nrc]

@danc: I was discussing with @jseyfried on irc. He suggested a simpler approach would be to add a single_step flag to MacroExpander. You could then check this and if true return after a single step, rather than recursing. Would that work for you?

Separately, there seems to be a limitation with either approach which is that after the first round of expansion, the expander will no longer be able to name macros which are in any module other than the crate root. This is because of the way the expander tracks modularisation of modules. I think we just live with that. It should fix itself once we re-jig macro name resolution. We can talk more on irc on Monday.

[bors]

☔ The latest upstream changes (presumably #34860) made this pull request unmergeable. Please resolve the merge conflicts.

[arielb1]

@bors r-``

[arielb1]

@bors r=nrc

[bors]

📌 Commit 866da14 has been approved by nrc

[arielb1]

@bors r-

[bors]

💥 Test timed out

[alexcrichton]

Unfortunately this now looks like it has some rebase conflicts, @DanielJCampbell could you rebase so we can re-r+?

[jseyfried]

nit: these arguments can be on one line.

[DanielJCampbell]

Fixed nits

[jseyfried]

One last thing -- user-defined extensions should be inserted before calling expander.load_macros(&items) so that macros imported from extern crates continue to shadow user-defined extensions (#[plugin]s). Otherwise, we might break code for no good reason.

[DanielJCampbell]

Ok, I'll also add a comment to that effect, since that seems worth a warning.

[jseyfried]

@DanielJCampbell looks good! Could you squash the two commits?

@nrc r=me unless you have other comments.

[nrc]

lgtm

[DanielJCampbell]

Squashed the commits together.

[jseyfried]

@bors r+

[bors]

📌 Commit 2fefe00 has been approved by jseyfried

[bors]

⌛ Testing commit 2fefe00 with merge 7637782...

[bors]

💔 Test failed - auto-win-msvc-64-opt-rustbuild

[DanielJCampbell]

Looks like I missed the tests, I'll fix that up shortly.

[jseyfried]

@bors r+

[bors]

📌 Commit 61c7569 has been approved by jseyfried

[bors]

⌛ Testing commit 61c7569 with merge 68d9284...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-cargotest, auto-linux-64-cross-freebsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-no-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-no-mir, auto-win-msvc-64-opt-rustbuild
Approved by: jseyfried
Pushing 68d9284 to master...