Remove TreeAndJoint in favor of joint field on the Token by matklad · Pull Request #64782 · rust-lang/rust

matklad · 2019-09-25T21:15:23Z

This is a step towards #63689.

The TL;RD of that issue is that rustc represents >> as a single token at the moment, while proc_macros represent it as a pair of tokens (>, Joint), (>, _). And we want to move the parser to proc_macro representation of tokens, with the two main motivations being: a) don't having two things in the compiler, b) making parser's interface more obviously right, to help with extracting the parser into a separate library.

Moreover, at the moment rustc actually tracks jointness in two ways in a single data structure. TokenStream, before this PR stores (TokenTree, IsJoin) pairs, while the TokenTree itself can be a composite or decomposed token. And this jointness info is pretty easily lost via this impl. This PR by itself doesn't solve the two reprs problem, but it does help with not accidentally loosing jointness. In particular, macros by example now preserve jointness, while they erased it previously.

rust-highfive · 2019-09-25T21:15:27Z

r? @eddyb

(rust_highfive has picked a reviewer for you, use r? to override)

rust-highfive · 2019-09-25T21:36:56Z

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-25T21:15:59.9574931Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-25T21:15:59.9763476Z ##[command]git config gc.auto 0
2019-09-25T21:15:59.9825361Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-25T21:15:59.9899145Z ##[command]git config --get-all http.proxy
2019-09-25T21:16:00.0038989Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64782/merge:refs/remotes/pull/64782/merge
---
2019-09-25T21:23:16.1754291Z Compiling serde_json v1.0.40
2019-09-25T21:23:18.0825857Z Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-09-25T21:23:29.7136508Z Finished release [optimized] target(s) in 1m 32s
2019-09-25T21:23:29.7232808Z tidy check
2019-09-25T21:23:30.6267858Z tidy error: /checkout/src/libsyntax/ext/mbe/quoted.rs:108: line longer than 100 chars
2019-09-25T21:23:30.6327901Z tidy error: /checkout/src/libsyntax/parse/parser.rs:909: TODO is deprecated; use FIXME
2019-09-25T21:23:30.6380387Z tidy error: /checkout/src/libsyntax/tokenstream.rs:362: line longer than 100 chars
2019-09-25T21:23:31.8434623Z some tidy checks failed
2019-09-25T21:23:31.8438896Z 2019-09-25T21:23:31.8438896Z 2019-09-25T21:23:31.8440216Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-09-25T21:23:31.8441240Z 2019-09-25T21:23:31.8441333Z 2019-09-25T21:23:31.8448732Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-09-25T21:23:31.8448934Z Build completed unsuccessfully in 0:01:35
2019-09-25T21:23:31.8448934Z Build completed unsuccessfully in 0:01:35
2019-09-25T21:23:31.8506334Z == clock drift check ==
2019-09-25T21:23:31.8527118Z local time: Wed Sep 25 21:23:31 UTC 2019
2019-09-25T21:23:32.1310304Z network time: Wed, 25 Sep 2019 21:23:32 GMT
2019-09-25T21:23:32.1312829Z == end clock drift check ==
2019-09-25T21:23:33.5161474Z ##[error]Bash exited with code '1'.
2019-09-25T21:23:33.5204735Z ##[section]Starting: Checkout
2019-09-25T21:23:33.5206752Z ==============================================================================
2019-09-25T21:23:33.5206810Z Task : Get sources
2019-09-25T21:23:33.5206878Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

matklad · 2019-09-26T06:49:24Z

Should be ready for review now. Note that this causes the size_of::<Token> to grow from 24 to 32. cc @petrochenkov

rust-highfive · 2019-09-26T08:27:32Z

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-26T06:45:51.9729379Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-26T06:45:51.9976725Z ##[command]git config gc.auto 0
2019-09-26T06:45:52.0015812Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-26T06:45:52.0064379Z ##[command]git config --get-all http.proxy
2019-09-26T06:45:52.0211024Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64782/merge:refs/remotes/pull/64782/merge
---
2019-09-26T07:49:58.7359057Z .................................................................................................... 1500/9044
2019-09-26T07:50:05.0697726Z .................................................................................................... 1600/9044
2019-09-26T07:50:17.9643339Z ..........................................................................i...............i......... 1700/9044
2019-09-26T07:50:24.9782328Z .................................................................................................... 1800/9044
2019-09-26T07:50:33.7381969Z .................................................................iiiii.............................. 1900/9044
2019-09-26T07:50:54.1296645Z .................................................................................................... 2100/9044
2019-09-26T07:50:56.7744491Z .................................................................................................... 2200/9044
2019-09-26T07:50:59.9748452Z .................................................................................................... 2300/9044
2019-09-26T07:51:08.7806412Z .................................................................................................... 2400/9044
---
2019-09-26T07:54:11.4616322Z ........................................................i...............i........................... 4700/9044
2019-09-26T07:54:20.8240165Z .................................................................................................... 4800/9044
2019-09-26T07:54:29.5503711Z .................................................................................................... 4900/9044
2019-09-26T07:54:37.1379409Z .................................................................................................... 5000/9044
2019-09-26T07:54:47.1648851Z ...........................................ii.ii.................................................... 5100/9044
2019-09-26T07:54:57.4469127Z .................................................................................................... 5300/9044
2019-09-26T07:55:07.8916501Z .................................................................................................... 5400/9044
2019-09-26T07:55:15.4166577Z ........i........................................................................................... 5500/9044
2019-09-26T07:55:21.0890346Z .................................................................................................... 5600/9044
2019-09-26T07:55:21.0890346Z .................................................................................................... 5600/9044
2019-09-26T07:55:33.1162441Z .................................................................................................... 5700/9044
2019-09-26T07:55:46.2989657Z ...ii...i..ii...........i........................................................................... 5800/9044
2019-09-26T07:56:08.6164520Z .................................................................................................... 6000/9044
2019-09-26T07:56:14.8227637Z .................................................................................................... 6100/9044
2019-09-26T07:56:14.8227637Z .................................................................................................... 6100/9044
2019-09-26T07:56:28.9016470Z .....i..ii.......................................................................................... 6200/9044
2019-09-26T07:56:48.4105556Z .................................................................i.................................. 6400/9044
2019-09-26T07:56:50.6791753Z .................................................................................................... 6500/9044
2019-09-26T07:56:53.2662670Z .....................................i.............................................................. 6600/9044
2019-09-26T07:56:57.4735396Z .................................................................................................... 6700/9044
---
2019-09-26T07:57:53.5278610Z .................................................................................................... 7200/9044
2019-09-26T07:57:59.2503146Z .................................................................................................... 7300/9044
2019-09-26T07:58:06.8505100Z .................................................................................................... 7400/9044
2019-09-26T07:58:17.3215250Z .................................................................................................... 7500/9044
2019-09-26T07:58:27.7458837Z .........................................................................................ii......i.. 7600/9044
2019-09-26T07:58:39.2573073Z .................................................................................................... 7800/9044
2019-09-26T07:58:57.1880621Z .................................................................................................... 7900/9044
2019-09-26T07:59:05.7054847Z .................................................................................................... 8000/9044
2019-09-26T07:59:16.2966610Z .................................................................................................... 8100/9044
---
2019-09-26T08:01:35.4031676Z finished in 5.575
2019-09-26T08:01:35.4250177Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:01:35.6283680Z 2019-09-26T08:01:35.6283901Z running 150 tests
2019-09-26T08:01:39.0102681Z i....iii......iii..iiii....i.............................i..i..................i....i.........ii.i.i 100/150
2019-09-26T08:01:41.1831786Z ..iiii..............i.........iii.i.......ii......
2019-09-26T08:01:41.1832781Z 2019-09-26T08:01:41.1842336Z finished in 5.759
2019-09-26T08:01:41.2042583Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:01:41.3755319Z ---
2019-09-26T08:01:43.5114359Z finished in 2.307
2019-09-26T08:01:43.5314492Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:01:43.6990749Z 2019-09-26T08:01:43.6990934Z running 9 tests
2019-09-26T08:01:43.6991799Z iiiiiiiii
2019-09-26T08:01:43.6992168Z 2019-09-26T08:01:43.6992236Z finished in 0.167
2019-09-26T08:01:43.7172787Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:01:43.9102287Z ---
2019-09-26T08:02:02.6186189Z finished in 18.901
2019-09-26T08:02:02.6413099Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:02:02.8322678Z 2019-09-26T08:02:02.8323617Z running 123 tests
2019-09-26T08:02:27.5537314Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-09-26T08:02:32.2637457Z i.i.i......iii.i.....ii
2019-09-26T08:02:32.2638761Z 2019-09-26T08:02:32.2642868Z finished in 29.622
2019-09-26T08:02:32.2650159Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:02:32.2650459Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-09-26T08:15:49.5279644Z 2019-09-26T08:15:49.5287122Z Doc-tests core
2019-09-26T08:15:54.6699660Z 2019-09-26T08:15:54.6700769Z running 2405 tests
2019-09-26T08:16:06.1708055Z ......iiiii......................................................................................... 100/2405
2019-09-26T08:16:17.1739036Z ...............................................................................ii................... 200/2405
2019-09-26T08:16:43.5271634Z .i.................................................................................................. 400/2405
2019-09-26T08:16:43.5271634Z .i.................................................................................................. 400/2405
2019-09-26T08:16:54.4771219Z ................................................i..i.................iiii........................... 500/2405
2019-09-26T08:17:15.5426492Z .................................................................................................... 700/2405
2019-09-26T08:17:26.3169965Z .................................................................................................... 800/2405
2019-09-26T08:17:37.1899614Z .................................................................................................... 900/2405
2019-09-26T08:17:48.0928386Z .................................................................................................... 1000/2405
---
2019-09-26T08:22:08.4525213Z 2019-09-26T08:22:08.4530951Z running 992 tests
2019-09-26T08:22:30.5153685Z i................................................................................................... 100/992
2019-09-26T08:22:42.5688477Z .................................................................................................... 200/992
2019-09-26T08:22:51.0853698Z .................iii......i......i...i......i....................................................... 300/992
2019-09-26T08:22:57.2114845Z .................................................................................................... 400/992
2019-09-26T08:23:05.2882830Z ...................................i..i.................................ii.......................... 500/992
2019-09-26T08:23:20.8279123Z .................................................................................................... 700/992
2019-09-26T08:23:20.8279123Z .................................................................................................... 700/992
2019-09-26T08:23:29.3880463Z ..................iiii.............................................................................. 800/992
2019-09-26T08:23:44.8190759Z .................................................................................................... 900/992
2019-09-26T08:23:52.7084225Z ........................................iiii................................................
2019-09-26T08:23:52.7086209Z 2019-09-26T08:23:52.7127786Z finished in 199.823
2019-09-26T08:23:52.7146359Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:23:52.9317840Z Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-09-26T08:27:19.1115624Z 2019-09-26T08:27:19.1176341Z finished in 18.682
2019-09-26T08:27:19.1201556Z Testing syntax stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:27:19.3430744Z Compiling syntax v0.0.0 (/checkout/src/libsyntax)
2019-09-26T08:27:25.7702186Z error[E0599]: no method named `joint` found for type `tokenstream::TokenTree` in the current scope
2019-09-26T08:27:25.7703884Z --> src/libsyntax/tokenstream/tests.rs:101:61
2019-09-26T08:27:25.7704551Z |
2019-09-26T08:27:25.7705069Z 101 | builder.push(TokenTree::token(token::Dot, sp(0, 1)).joint());
2019-09-26T08:27:25.7705701Z | ^^^^^ method not found in `tokenstream::TokenTree`
2019-09-26T08:27:25.7706713Z ::: src/libsyntax/tokenstream.rs:44:1
2019-09-26T08:27:25.7707168Z |
2019-09-26T08:27:25.7707660Z 44 | pub enum TokenTree {
2019-09-26T08:27:25.7707660Z 44 | pub enum TokenTree {
2019-09-26T08:27:25.7708616Z | ------------------ method `joint` not found for this
2019-09-26T08:27:25.7709270Z 2019-09-26T08:27:25.7740029Z error[E0599]: no method named `joint` found for type `tokenstream::TokenTree` in the current scope
2019-09-26T08:27:25.7740802Z --> src/libsyntax/tokenstream/tests.rs:102:61
2019-09-26T08:27:25.7741370Z |
2019-09-26T08:27:25.7741993Z 102 | builder.push(TokenTree::token(token::Dot, sp(1, 2)).joint());
2019-09-26T08:27:25.7744095Z | ^^^^^ method not found in `tokenstream::TokenTree`
2019-09-26T08:27:25.7745326Z ::: src/libsyntax/tokenstream.rs:44:1
2019-09-26T08:27:25.7745829Z |
2019-09-26T08:27:25.7746198Z 44 | pub enum TokenTree {
2019-09-26T08:27:25.7746198Z 44 | pub enum TokenTree {
2019-09-26T08:27:25.7746765Z | ------------------ method `joint` not found for this
2019-09-26T08:27:26.8460622Z error: aborting due to 2 previous errors
2019-09-26T08:27:26.8460733Z 2019-09-26T08:27:26.8460979Z For more information about this error, try `rustc --explain E0599`.
2019-09-26T08:27:26.9297108Z error: could not compile `syntax`.
---
2019-09-26T08:27:26.9382491Z == clock drift check ==
2019-09-26T08:27:26.9401598Z local time: Thu Sep 26 08:27:26 UTC 2019
2019-09-26T08:27:27.0400058Z network time: Thu, 26 Sep 2019 08:27:27 GMT
2019-09-26T08:27:27.0400310Z == end clock drift check ==
2019-09-26T08:27:27.6673882Z ##[error]Bash exited with code '1'.
2019-09-26T08:27:27.6727832Z ##[section]Starting: Checkout
2019-09-26T08:27:27.6730247Z ==============================================================================
2019-09-26T08:27:27.6730324Z Task : Get sources
2019-09-26T08:27:27.6730376Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

matklad · 2019-09-26T11:16:28Z

I wonder if we should go further already in this PR, and push jointness further down, to the relevant TokenKinds?

eddyb

LGTM

eddyb · 2019-09-26T13:40:32Z

r? @petrochenkov cc @nnethercote (on the size change)

petrochenkov · 2019-09-26T19:31:01Z

@bors try @rust-timer queue

rust-timer · 2019-09-26T19:31:03Z

Awaiting bors try build completion

bors · 2019-09-26T19:31:16Z

⌛ Trying commit 42f0795 with merge 9ee6b923ba02846e7df5357e4f94b5d8e0126f0c...

petrochenkov · 2019-09-26T19:34:43Z

 static_assert_size!(TokenKind, 16);

+#[derive(Clone, Copy, PartialEq, RustcEncodable, RustcDecodable, Debug)]
+pub enum IsJoint {


This can almost reuse proc_macro::Spacing, except that it doesn't implement Rustc{Encodable,Decodable}.
Perhaps it can still be done if the *codable traits are implemented for Token manually?

I definitely think that we should at least rename this to Spacing. As for reusing the actual type, I don't know. In general, I feel that that a boilerplaty conversion on the boundary between two systems is better than sharing actual types, as it is more resilient to changes.

At least proc_macro::Spacing is stable and won't change though.

except that it doesn't implement Rustc{Encodable,Decodable}
Actually the impls can be added to libserialize, like it's done for some libstd types used in the compiler in https://github.com/rust-lang/rust/blob/master/src/libserialize/collection_impls.rs.

Anyway, at least the naming should be consistent with proc_macro::Spacing (and let's also remove pub use IsJoint::*;).

petrochenkov · 2019-09-26T19:36:26Z

I wonder if we should go further already in this PR, and push jointness further down, to the relevant TokenKinds?

I don't think that's a good idea, we should probably track jointness for all tokens, even if we don't expose it for all token kinds in the proc macro API.
We can still use it for diagnostics or more precise pretty-printing.

petrochenkov · 2019-09-26T19:37:24Z

 self.real_token();
- let is_joint = self.joint_to_prev == Joint && self.token.is_op();
- Ok((tt, if is_joint { Joint } else { NonJoint }))
+ let is_joint = self.joint_to_prev == Joint && token.is_op() && self.token.is_op();


Perhaps it's time to move the is_op check to the proc macro API boundary (and track precise jointness for everything internally)?

Makes sense. I've modified this line to avoid modifying a unit test, but I don't see why can't we move it entirely to proc-macro layer. (although I'd rather do it in a separate PR).

petrochenkov · 2019-09-26T19:39:25Z

Note that this causes the size_of:: to grow from 24 to 32.

We should be able to recover this later, the largest thing in Token is currently Lit, which can be split into two tokens (base and suffix) if we track jointness for all tokens.

matklad · 2019-09-26T19:44:23Z

which can be split into two tokens (base and suffix) if we track jointness for all tokens.

I am not entirely sure that this is a good idea. My end goal here is to use exactly the same token trees as proc macros. And, in proc-macro model, suffix is a part of a literal, and not a separate joint token. OTOH, due to the way macro_rules work, we might be unable to move to proc macro model fully anyway...

bors · 2019-09-26T22:23:22Z

☀️ Try build successful - checks-azure
Build commit: 9ee6b923ba02846e7df5357e4f94b5d8e0126f0c (9ee6b923ba02846e7df5357e4f94b5d8e0126f0c)

rust-timer · 2019-09-26T22:23:24Z

Queued 9ee6b923ba02846e7df5357e4f94b5d8e0126f0c with parent ddf4386, future comparison URL.

rust-timer · 2019-09-27T03:10:11Z

Finished benchmarking try commit 9ee6b923ba02846e7df5357e4f94b5d8e0126f0c, comparison URL.

nnethercote · 2019-09-27T03:43:59Z

Slight increases in instruction counts, up to 0.5%. Unsurprising that html5ever is the worst, it's very heavy on TokenTree stuff. Increasing the size of Token is a shame, especially given that it increases by 8 bytes to store only a single bit of extra information.

There is no description of the motivation for this change, either on the PR or the commit. I've looked at the commit and see that it moves a bunch of stuff around, but the benefits of doing that aren't clear to me.

@matklad, you said above "My end goal here is to use exactly the same token trees as proc macros." Is that the motivation? Can you explain that some more? Thanks.

matklad · 2019-09-27T07:06:06Z

@nnethercote sorry for the confusing description. This is a step towards #63689. I put this into the comment in the code, but should have duplicated this into the commit description as well, will do with the next rebase.

The TL;RD of that issue is that rustc represents >> as a single token at the moment, while proc_macros represent it as a pair of tokens (>, Joint), (>, _). And we want to move the parser to proc_macro representation of tokens, with the two main motivations being: a) don't having two things in the compiler, b) making parser's interface more obviously right, to help with extracting the parser into a separate library.

Moreover, at the moment rustc actually tracks jointness in two ways in a single data structure. TokenStream, before this PR stores (TokenTree, IsJoin) pairs, while the TokenTree itself can be a composite or decomposed token. And this jointness info is pretty easily lost via this impl. This PR by itself doesn't solve the two reprs problem, but it does help with not accidentally loosing jointness. In particular, macros by example now preserve jointness, while they erased it previously.

I don't know whether the refactoring here is worth the perf loss. Is there perhaps some easy way to recover ground here?

eddyb · 2019-09-27T07:10:00Z

I put this into the comment in the code, but should have duplicated this into the commit description as well, will do with the next rebase.

Please edit the PR description as well, that's what most people look at.

nnethercote · 2019-09-27T08:13:09Z

I suspect the Token size increase accounts for most of the perf effect. I don't know if there's a way to squeeze that single bit into Token without increasing its size. (I've been contemplating the same question with ParamEnv, another frequently-used type that is bloated by 8 bytes due to a boolean flag.)

petrochenkov · 2019-09-27T09:50:24Z

+ Token { kind, span, joint: NonJoint }
+ }
+
+ crate fn with_joint(self, joint: IsJoint) -> Self {


From the uses of this function it seems like a usual constructor fn with_spacing(TokenKind, Span, Spacing) -> Self would fit better.

petrochenkov · 2019-09-27T10:09:42Z

The perf diff seems negligible and the change is something needs to be done anyway.
Note that any token that is a part of a token stream doesn't get larger because it previously was inside of TokenAndJoint.
So, the regression should only affect isolated tokens, which are not commonly used, AFAIK.

matklad · 2019-09-27T10:16:34Z

@petrochenkov I think the regression may also affect macro_by_example, precisely because mbe::TokenStream was jointness-less before this PR. Quick manual checking however shows that the size of mbe::TokenTree stays the same.

JohnCSimon · 2019-10-05T18:19:34Z

Ping from triage.
This PR has sat idle for the last 8 days
@matklad Can you please address the merge conflict?
CC: @petrochenkov

Thank you!

matklad · 2019-10-05T22:13:03Z

Yeah, I am back from dev fest Toulouse and will push this over the finish line next week

JohnCSimon · 2019-10-12T03:54:29Z

Ping from triage.

@matklad Hi! Just letting you know this still needs to be rebased.
Thanks.

nnethercote · 2019-10-15T05:47:55Z

@matklad: #65261 landed and caused additional conflicts, but they should be fairly easy to resolve. I have another set of TokenStream changes ready but I am happy to let you land this first.

matklad · 2019-10-15T05:51:56Z

@nnethercote thanks for the heads up! I turned out to be pretty busy this moment, so please go ahead with your other changes, and I'll rebase this work when I come back to this. Closing for know, but I am keeping one eye on it!

rust-highfive assigned eddyb Sep 25, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 25, 2019

matklad force-pushed the token-joint branch 3 times, most recently from a3da6f9 to ed58f79Compare September 26, 2019 06:45

matklad changed the title ~~WIP: remove TreeAndJoint in favor of joint field on the Token~~Remove TreeAndJoint in favor of joint field on the TokenSep 26, 2019

matklad force-pushed the token-joint branch 3 times, most recently from 5a73379 to a31bff6Compare September 26, 2019 10:02

Move jointness info from TokenStream to Token
42f0795

matklad force-pushed the token-joint branch from a31bff6 to 42f0795Compare September 26, 2019 10:36

eddyb approved these changes Sep 26, 2019
View reviewed changes

rust-highfive assigned petrochenkov and unassigned eddyb Sep 26, 2019

petrochenkov reviewed Sep 26, 2019
View reviewed changes

petrochenkov reviewed Sep 27, 2019
View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 27, 2019

nnethercote mentioned this pull request Oct 7, 2019
TokenStream manipulations are 1000x too slow #65080
Closed

matklad closed this Oct 15, 2019

matklad mentioned this pull request Aug 14, 2020
WIP: Move jointness info from TokenStream to Token #75528
Closed

Uh oh!

Uh oh!

Conversation

matklad commented Sep 25, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented Sep 25, 2019

Uh oh!

rust-highfive commented Sep 25, 2019

Uh oh!

matklad commented Sep 26, 2019

Uh oh!

rust-highfive commented Sep 26, 2019

Uh oh!

matklad commented Sep 26, 2019

Uh oh!

eddyb left a comment

Choose a reason for hiding this comment

Uh oh!

eddyb commented Sep 26, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Sep 26, 2019

Uh oh!

rust-timer commented Sep 26, 2019

Uh oh!

bors commented Sep 26, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented Sep 26, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented Sep 26, 2019

Uh oh!

matklad commented Sep 26, 2019

Uh oh!

bors commented Sep 26, 2019

Uh oh!

rust-timer commented Sep 26, 2019

Uh oh!

rust-timer commented Sep 27, 2019

Uh oh!

nnethercote commented Sep 27, 2019

Uh oh!

matklad commented Sep 27, 2019

Uh oh!

eddyb commented Sep 27, 2019

Uh oh!

nnethercote commented Sep 27, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented Sep 27, 2019

Uh oh!

matklad commented Sep 27, 2019

Uh oh!

JohnCSimon commented Oct 5, 2019

Uh oh!

matklad commented Oct 5, 2019

Uh oh!

JohnCSimon commented Oct 12, 2019

Uh oh!

nnethercote commented Oct 15, 2019

Uh oh!

matklad commented Oct 15, 2019

Uh oh!

Reviewers

Assignees

matklad commented Sep 25, 2019•
edited
Loading

eddyb commented Sep 26, 2019•
edited
Loading