Skip to content

More thread-safety changes - #49390

Merged
bors merged 10 commits into
rust-lang:masterfrom
Zoxc:sync-syntax
Apr 10, 2018
Merged

More thread-safety changes#49390
bors merged 10 commits into
rust-lang:masterfrom
Zoxc:sync-syntax

Conversation

@Zoxc

@ZoxcZoxc commented Mar 26, 2018

Copy link
Copy Markdown
Contributor

@emilyalbiniemilyalbini added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 27, 2018
@michaelwoerister

michaelwoerister commented Mar 27, 2018

Copy link
Copy Markdown
Member

@rust-lang/compiler Does anybody know what LazyTokenStream is? It seems to be unused.

@petrochenkov

petrochenkov commented Mar 27, 2018•
edited
Loading

Copy link
Copy Markdown
Contributor

@michaelwoerister

Does anybody know what LazyTokenStream is?

It's a hack! But it's used.
Many nonterminal tokens (e.g. NtExpr) don't know their original token streams from which they were parsed (they have only AST fragments like ast::Expr), but these token streams are sometimes needed (for example if they are a part of macro definition and that macro definition needs to be stored in metadata), so AST fragments are pretty-printed into text (!) and then parsed back into the token stream (on best-effort basis).
LazyTokenStream serves as a cache for these AST -> TokenStream conversions to avoid doing these conversions multiple times, so removing it won't affect correctness, only performance.

@Zoxc

Zoxc commented Mar 28, 2018

Copy link
Copy Markdown
ContributorAuthor

@petrochenkov Why do we keep the AST around in the token stream though? It seems more correct to use the original TokenStream instead of relying on an lossy AST -> TokenStream transformation

@petrochenkov

petrochenkov commented Mar 28, 2018

Copy link
Copy Markdown
Contributor

@Zoxc

Why do we keep the AST around in the token stream though?

Who knows. Nonterminal is represented with AST fragments since at least 2012, years before token streams were introduced.
Someone needs to port expansion code working with nonterminals to token trees.

@shepmaster

shepmaster commented Apr 7, 2018

Copy link
Copy Markdown
Member

Ping from triage, @michaelwoerister !

michaelwoerister
michaelwoerister reviewed Apr 9, 2018
View reviewed changes
Comment threadsrc/librustc/hir/map/mod.rs
pub fn intern_inlined_body(&self, def_id: DefId, body: Body) -> &'hir Body {
let mut inlined_bodies = self.inlined_bodies.borrow_mut();
if let Some(&b) = inlined_bodies.get(&def_id) {
debug_assert_eq!(&body, b);

@michaelwoeristermichaelwoeristerApr 9, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can this condition hold? body is on the stack and b is in the arena. Am I overlooking something?

@ZoxcZoxcApr 9, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a pointer comparison. This check ensures that we always give the same body for the same def_id to intern_inlined_body.

@michaelwoeristermichaelwoeristerApr 10, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you're right of course :)

@michaelwoerister

michaelwoerister commented Apr 9, 2018

Copy link
Copy Markdown
Member

@Zoxc, looks good to me except for intern_inlined_body. I think you can just remove the assertion there.

@michaelwoerister

michaelwoerister commented Apr 10, 2018

Copy link
Copy Markdown
Member

@bors r+

@bors

bors commented Apr 10, 2018

Copy link
Copy Markdown
Collaborator

📌 Commit 1e3f638 has been approved by michaelwoerister

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 10, 2018
@bors

bors commented Apr 10, 2018

Copy link
Copy Markdown
Collaborator

⌛ Testing commit 1e3f638 with merge 67712d7...

bors added a commit that referenced this pull request Apr 10, 2018
@bors
Auto merge of#49390- Zoxc:sync-syntax, r=michaelwoerister
67712d7
More thread-safety changes
r? @michaelwoerister
@bors

bors commented Apr 10, 2018

Copy link
Copy Markdown
Collaborator

☀️ Test successful - status-appveyor, status-travis
Approved by: michaelwoerister
Pushing 67712d7 to master...

@bors
bors merged commit 1e3f638 into rust-lang:masterApr 10, 2018
@borsbors mentioned this pull request Apr 10, 2018
Closed
This was referenced Apr 10, 2018
Merged
Closed
Merged
Merged
Merged
@Zoxc
Zoxc deleted the sync-syntax branch April 10, 2018 12:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@michaelwoeristermichaelwoeristermichaelwoerister left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@michaelwoeristermichaelwoerister

Labels

S-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Zoxc@michaelwoerister@petrochenkov@shepmaster@bors@emilyalbini