Rebase rustc-rayon on rayon-1.2 by cuviper · Pull Request #64358 · rust-lang/rust

cuviper · 2019-09-10T17:15:26Z

See also rust-lang/rustc-rayon#3

rust-highfive · 2019-09-10T17:15:36Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

cuviper · 2019-09-10T17:20:50Z

Let's try full CI to exercise parallel queries...

@bors try

bors · 2019-09-10T17:20:55Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout rustc-rayon-1.2(switch to your branch)
git fetch upstream master(retrieve the latest master)
git rebase upstream/master -p(rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self rustc-rayon-1.2 --force-with-lease(update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging Cargo.lock
CONFLICT (content): Merge conflict in Cargo.lock
Automatic merge failed; fix conflicts and then commit the result.

Mark-Simulacrum · 2019-09-10T17:22:21Z

Ah, bors try is not going to run full CI at this point (and we don't actually have a parallel test builder, only a check builder for --cfg parallel)

cuviper · 2019-09-10T17:25:18Z

Ah, bors try is not going to run full CI at this point

Just because of the merge conflict? I rebased again for Cargo.lock now.

(and we don't actually have a parallel test builder, only a check builder for --cfg parallel)

How have other parallel PRs been tested? Should I flip it on somewhere just to try it out?

cuviper · 2019-09-10T17:29:21Z

@RalfJung was excited to get rid of this -- rust-lang/rustc-rayon#3 (comment)

Yes, this is great, thanks :)

rust-highfive · 2019-09-10T17:42:35Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-10T17:23:13.0157544Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-10T17:23:13.0356705Z ##[command]git config gc.auto 0
2019-09-10T17:23:13.0434208Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-10T17:23:13.0486354Z ##[command]git config --get-all http.proxy
2019-09-10T17:23:13.0645168Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64358/merge:refs/remotes/pull/64358/merge
---
2019-09-10T17:26:54.7081705Z ################################################### 71.4%
2019-09-10T17:26:54.7428324Z ##################################################################### 96.4%
2019-09-10T17:26:54.7430784Z ######################################################################## 100.0%
2019-09-10T17:26:55.1480572Z extracting /checkout/obj/build/cache/2019-08-13/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
2019-09-10T17:26:55.2112365Z Updating git repository `https://github.com/cuviper/rayon`
---
2019-09-10T17:30:29.7855488Z * highest error code: E0733
2019-09-10T17:30:30.1432526Z * 263 features
2019-09-10T17:30:30.8030413Z Dependencies not on the whitelist:
2019-09-10T17:30:30.8031282Z * crossbeam-queue 2019-09-10T17:30:30.8031346Z invalid source: "git+https://github.com/cuviper/rayon?branch=rustc#7ffde12e0cc7b5ab2ed8bea8c6c08c3005ac4db7"
2019-09-10T17:30:30.8031423Z invalid source: "git+https://github.com/cuviper/rayon?branch=rustc#7ffde12e0cc7b5ab2ed8bea8c6c08c3005ac4db7"
2019-09-10T17:30:30.8495597Z some tidy checks failed
2019-09-10T17:30:30.8501469Z 2019-09-10T17:30:30.8501469Z 2019-09-10T17:30:30.8505513Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-09-10T17:30:30.8505732Z 2019-09-10T17:30:30.8505808Z 2019-09-10T17:30:30.8519880Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-09-10T17:30:30.8520357Z Build completed unsuccessfully in 0:01:35
2019-09-10T17:30:30.8520357Z Build completed unsuccessfully in 0:01:35
2019-09-10T17:30:30.8580734Z == clock drift check ==
2019-09-10T17:30:30.8600062Z local time: Tue Sep 10 17:30:30 UTC 2019
2019-09-10T17:30:31.1403598Z network time: Tue, 10 Sep 2019 17:30:31 GMT
2019-09-10T17:30:31.1407200Z == end clock drift check ==
2019-09-10T17:30:32.4425298Z ##[error]Bash exited with code '1'.
2019-09-10T17:30:32.4461074Z ##[section]Starting: Checkout
2019-09-10T17:30:32.4463317Z ==============================================================================
2019-09-10T17:30:32.4463366Z Task : Get sources
2019-09-10T17:30:32.4463411Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

cuviper · 2019-09-10T17:43:48Z

Tidy doesn't like my (temporary) git source replacement, which is understandable.

Zoxc · 2019-09-11T01:49:07Z

@cuviper I usually just test the PRs locally with [rust] parallel-compiler = true in config.toml.

cuviper · 2019-09-12T19:35:27Z

OK, well it looks pretty good locally on x86_64-unknown-linux-gnu. I just pushed a fix for thread names that showed up in UI tests. There's one more failure that I'm not sure about:

---- [ui] ui/traits/cycle-cache-err-60010.rs stdout ----
diff of stderr:
6 |
7 = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
8
- error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
- --> $DIR/cycle-cache-err-60010.rs:30:6
- |
- LL | impl Database for RootDatabase {
- | ^^^^^^^^
- |
- = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
- = note: required because it appears within the type `SalsaStorage`
-
- error: aborting due to 2 previous errors
+ error: aborting due to previous error
19
20 For more information about this error, try `rustc --explain E0275`.
21
The actual stderr differed from the expected stderr.
Actual stderr saved to /home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010/cycle-cache-err-60010.stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args traits/cycle-cache-err-60010.rs`
error: 1 errors occurred comparing output.
status: exit code: 1
command: "/home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/home/jistone/rust/rust/src/test/ui/traits/cycle-cache-err-60010.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/jistone/rust/rust/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010/auxiliary" "-A" "unused"
stdout:
------------------------------------------
------------------------------------------
stderr:
------------------------------------------
error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
--> /home/jistone/rust/rust/src/test/ui/traits/cycle-cache-err-60010.rs:27:5
|
LL | _parse: <ParseQuery as Query<RootDatabase>>::Data, //~ ERROR overflow
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0275`.
------------------------------------------

rust-highfive · 2019-09-12T19:54:31Z

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-12T19:35:42.9378034Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-12T19:35:42.9643729Z ##[command]git config gc.auto 0
2019-09-12T19:35:42.9727403Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-12T19:35:42.9798495Z ##[command]git config --get-all http.proxy
2019-09-12T19:35:42.9981277Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64358/merge:refs/remotes/pull/64358/merge
---
2019-09-12T19:39:23.7782859Z ############################### 43.5%
2019-09-12T19:39:23.8038159Z ############################################################# 85.6%
2019-09-12T19:39:23.8039019Z ######################################################################## 100.0%
2019-09-12T19:39:24.3694008Z extracting /checkout/obj/build/cache/2019-08-13/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
2019-09-12T19:39:24.4366313Z Updating git repository `https://github.com/cuviper/rayon`
---
2019-09-12T19:42:57.7267748Z * highest error code: E0733
2019-09-12T19:42:58.0903070Z * 263 features
2019-09-12T19:42:58.7504238Z Dependencies not on the whitelist:
2019-09-12T19:42:58.7505134Z * crossbeam-queue 2019-09-12T19:42:58.7505212Z invalid source: "git+https://github.com/cuviper/rayon?branch=rustc#7ffde12e0cc7b5ab2ed8bea8c6c08c3005ac4db7"
2019-09-12T19:42:58.7505295Z invalid source: "git+https://github.com/cuviper/rayon?branch=rustc#7ffde12e0cc7b5ab2ed8bea8c6c08c3005ac4db7"
2019-09-12T19:42:58.8026661Z some tidy checks failed
2019-09-12T19:42:58.8026768Z 2019-09-12T19:42:58.8026768Z 2019-09-12T19:42:58.8027586Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-09-12T19:42:58.8027686Z 2019-09-12T19:42:58.8027729Z 2019-09-12T19:42:58.8036443Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-09-12T19:42:58.8036519Z Build completed unsuccessfully in 0:01:33
2019-09-12T19:42:58.8036519Z Build completed unsuccessfully in 0:01:33
2019-09-12T19:42:58.8102601Z == clock drift check ==
2019-09-12T19:42:58.8132132Z local time: Thu Sep 12 19:42:58 UTC 2019
2019-09-12T19:42:58.8904919Z network time: Thu, 12 Sep 2019 19:42:58 GMT
2019-09-12T19:42:58.8905484Z == end clock drift check ==
2019-09-12T19:43:00.3323363Z ##[error]Bash exited with code '1'.
2019-09-12T19:43:00.3383753Z ##[section]Starting: Checkout
2019-09-12T19:43:00.3386141Z ==============================================================================
2019-09-12T19:43:00.3386206Z Task : Get sources
2019-09-12T19:43:00.3386262Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Zoxc · 2019-09-13T07:41:50Z

@cuviper That's a preexisting failure. (and so is the thread names, but fixing those doesn't hurt)

bors · 2019-09-14T19:06:47Z

☔ The latest upstream changes (presumably #64456) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · 2019-09-16T14:11:21Z

r? @cuviper

(feel free to reassign back to me or someone else once this is ready to go)

hdhoang · 2019-09-27T01:39:02Z

ping from triage @cuviper, how is progress on this PR?

cuviper · 2019-09-27T20:07:27Z

Waiting on rust-lang/rustc-rayon#3 -- I'll ping there.

JohnCSimon · 2019-10-05T18:33:16Z

Ping from triage.
@cuviper This PR has sat idle for the last 8 days - I see this as blocked from rust-lang/rustc-rayon#3
Can you address this?

Thanks.

See also rust-lang/rustc-rayon#3

cuviper · 2019-10-07T20:25:50Z

r? @nikomatsakis

The old stuff is still referenced by rustc-ap-rustc_data_structures, but I think that may be resolved once the real rustc_data_structures lands here, so a new one is published...

nikomatsakis · 2019-10-07T20:40:51Z

r=me

nikomatsakis · 2019-10-07T20:41:07Z

@bors delegate=cuviper

bors · 2019-10-07T20:41:08Z

✌️ @cuviper can now approve this pull request

cuviper · 2019-10-07T20:57:40Z

@bors r=nikomatsakis

bors · 2019-10-07T20:57:41Z

📌 Commit 29cd7eb has been approved by nikomatsakis

bors · 2019-10-07T21:24:35Z

⌛ Testing commit 29cd7eb with merge c20654e...

Rebase rustc-rayon on rayon-1.2 See also rust-lang/rustc-rayon#3

bors · 2019-10-08T01:05:01Z

☀️ Test successful - checks-azure
Approved by: nikomatsakis
Pushing c20654e to master...

RalfJung · 2019-10-18T08:57:07Z

The old stuff is still referenced by rustc-ap-rustc_data_structures, but I think that may be resolved once the real rustc_data_structures lands here, so a new one is published...

So what does it take to bump these? They currently keep a lot of old crates in our dependency tree:

[[package]]
name = "rustc-ap-rustc_data_structures"
version = "606.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca545744a5a9b42e3d0410d6290d40de96dd567253fe77f310c1de4afd213dd4"
dependencies = [
"cfg-if",
"crossbeam-utils 0.6.5",
"ena",
"indexmap",
"jobserver",
"lazy_static 1.3.0",
"log",
"parking_lot 0.9.0",
"rustc-ap-graphviz",
"rustc-ap-rustc_index",
"rustc-ap-serialize",
"rustc-hash",
"rustc-rayon 0.2.0",
"rustc-rayon-core 0.2.0",
"smallvec",
"stable_deref_trait",
]

RalfJung · 2019-10-18T09:02:51Z

Ah, looks like rustfmt needs a bump.

mati865 · 2019-10-18T10:35:20Z

Ah, looks like rustfmt needs a bump .

It's more complicated.
Only single version of rustc-ap-* can exist in Cargo.lock and it's pulled by racer (RLS dependency) and Rustfmt.
In the end it has to be coordinated among 3 different repositories.

rust-highfive assigned alexcrichton Sep 10, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 10, 2019

cuviper mentioned this pull request Sep 10, 2019
Rebase on rayon-1.2.0 and rayon-core-1.6.0 rust-lang/rustc-rayon#3
Merged

cuviper force-pushed the rustc-rayon-1.2 branch from 718109e to 465be65Compare September 10, 2019 17:22

cuviper commented Sep 10, 2019
View reviewed changes

rust-highfive assigned cuviper and unassigned alexcrichton Sep 16, 2019

hdhoang added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 27, 2019

cuviper mentioned this pull request Oct 3, 2019
UI Tests that check ICEing thread names break with parallel-compiler = true#65047
Closed

JohnCSimon added the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label Oct 5, 2019

cuviper added 2 commits October 7, 2019 13:20

Rebase rustc-rayon on rayon-1.2
33c4125
See also rust-lang/rustc-rayon#3

Name the threads in rayon's pool
fa2acbe

cuviper force-pushed the rustc-rayon-1.2 branch from 4152abe to fa2acbeCompare October 7, 2019 20:24

cuviper changed the title ~~[WIP] Rebase rustc-rayon on rayon-1.2~~Rebase rustc-rayon on rayon-1.2Oct 7, 2019

cuviper marked this pull request as ready for review October 7, 2019 20:25

rust-highfive assigned nikomatsakis and unassigned cuviper Oct 7, 2019

nikomatsakis approved these changes Oct 7, 2019
View reviewed changes

add crossbeam-queue to the whitelist
98779ed

This comment has been minimized.
Sign in to view

Update other rayon uses to 1.2 too
29cd7eb

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 7, 2019

bors added a commit that referenced this pull request Oct 7, 2019
Auto merge of#64358- cuviper:rustc-rayon-1.2, r=nikomatsakis
c20654e
Rebase rustc-rayon on rayon-1.2 See also rust-lang/rustc-rayon#3

bors added the merged-by-bors This PR was explicitly merged by bors. label Oct 8, 2019

bors merged commit 29cd7eb into rust-lang:masterOct 8, 2019

cuviper deleted the rustc-rayon-1.2 branch April 3, 2020 18:39

Uh oh!

Uh oh!

Conversation

cuviper commented Sep 10, 2019

Uh oh!

rust-highfive commented Sep 10, 2019

Uh oh!

cuviper commented Sep 10, 2019

Uh oh!

bors commented Sep 10, 2019

Uh oh!

Mark-Simulacrum commented Sep 10, 2019

Uh oh!

cuviper commented Sep 10, 2019

Uh oh!

cuviperSep 10, 2019

Choose a reason for hiding this comment

Uh oh!

RalfJungSep 10, 2019

Choose a reason for hiding this comment

Uh oh!

rust-highfive commented Sep 10, 2019

Uh oh!

cuviper commented Sep 10, 2019

Uh oh!

Zoxc commented Sep 11, 2019

Uh oh!

cuviper commented Sep 12, 2019

Uh oh!

rust-highfive commented Sep 12, 2019

Uh oh!

Zoxc commented Sep 13, 2019

Uh oh!

bors commented Sep 14, 2019

Uh oh!

alexcrichton commented Sep 16, 2019

Uh oh!

hdhoang commented Sep 27, 2019

Uh oh!

cuviper commented Sep 27, 2019

Uh oh!

JohnCSimon commented Oct 5, 2019

Uh oh!

cuviper commented Oct 7, 2019

Uh oh!

nikomatsakis commented Oct 7, 2019

Uh oh!

nikomatsakis commented Oct 7, 2019

Uh oh!

bors commented Oct 7, 2019

Uh oh!

This comment has been minimized.

cuviper commented Oct 7, 2019

Uh oh!

bors commented Oct 7, 2019

Uh oh!

bors commented Oct 7, 2019

Uh oh!

bors commented Oct 8, 2019

Uh oh!

RalfJung commented Oct 18, 2019

Uh oh!

RalfJung commented Oct 18, 2019

Uh oh!

mati865 commented Oct 18, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants