Add support for using a jobserver with Rayon

[Zoxc]

The Rayon changes are here: Zoxc/rayon#2

cc @alexcrichton
r? @nikomatsakis

[rust-highfive]

The job mingw-check of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:053ee1e0:start=1545120573675720183,finish=1545120574678870273,duration=1003150090
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=mingw-check
---
[00:03:25] Checking arena v0.0.0 (/checkout/src/libarena)
[00:03:25] Checking syntax_pos v0.0.0 (/checkout/src/libsyntax_pos)
[00:03:26] Checking rustc_errors v0.0.0 (/checkout/src/librustc_errors)
[00:03:39] Checking syntax_ext v0.0.0 (/checkout/src/libsyntax_ext)
[00:03:43] error[E0425]: cannot find function `continue_unblocked` in module `rayon_core`
[00:03:43] --> src/librustc/ty/query/job.rs:224:21
[00:03:43] 224 | rayon_core::continue_unblocked();
[00:03:43] | ^^^^^^^^^^^^^^^^^^ not found in `rayon_core`
[00:03:43] [00:04:02] error: aborting due to previous error
[00:04:02] error: aborting due to previous error
[00:04:02] [00:04:02] For more information about this error, try `rustc --explain E0425`.
[00:04:02] error: Could not compile `rustc`.
[00:04:02] [00:04:02] To learn more, run the command again with --verbose.
[00:04:02] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:04:02] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
[00:04:02] Build completed unsuccessfully in 0:03:01
travis_time:end:2f018950:start=1545120583159960648,finish=1545120825707478576,duration=242547517928
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 1.
---
travis_time:end:1a302ae4:start=1545120826103270363,finish=1545120826107763497,duration=4493134
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0012bf28
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:00b8ee70
travis_time:start:00b8ee70
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0c674736
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[alexcrichton]

I'm not too familiar with rayon internals, but can you describe at a high level the strategy for managing the jobserver tokens?

[Zoxc]

The strategy is to call Proxy::return_token before blocking and Proxy::acquire_token when blocking is complete (on both the main thread and in Rayon worker threads). The Proxy type papers over the differences between the implicit token given to the main thread and Acquired tokens, allowing the main thread to give its token over to the thread pool.

[alexcrichton]

As someone not familiar with rayon, can you expand on that a bit more? Acquiring and releasing a token is a pretty expensive operation and would be pretty inefficient if we did it super commonly, so is blocking in rayon something that's amortized over a long time?

[Zoxc]

There are 2 places where blocking can happen which is not related to setup/teardown:

• When a Rayon worker thread has no work to do and it has been spinning a bit looking for work. This can be reduced by ensuring that there's always work available for the thread pool. Making the compiler on-demand driven will help exposing parallelism to Rayon. This spinning isn't ideal if some other thread has work to do, but requires a token. We could probably adapt it to not spin in that case.

• When a query requires another query which is already executing in another thread. This can be eliminated by using fibers / coroutines so we can do something else instead of blocking.

I'd want to add that we spawn n threads immediately for the Rayon thread pool and they all will call Proxy::acquire_token before they do any real work. If they get a token they'll just spin a bit waiting for work then release the token and fall asleep. There is currently no parallelism in the front end of the compiler, so there won't be any work for them to do.

[Zoxc]

cc @michaelwoerister too

[michaelwoerister]

Could the jobserver be made abstract to Rayon? I.e. Rayon would not directly use the jobserver crate and instead just use something with the jobserver interface to acquire and release tokens? That would be a bit more general and rustc (or any other piece of code using Rayon) could do some buffering and application-specific management of tokens (e.g. keeping tokens around a bit longer if it expects for more work to show up soon).

Another approach that might be interesting: Add some way to tell Rayon the target number of active threads it should be running. It would then internally try to match this soft target by not assigning any more work to threads it wants to wind down.

[nikomatsakis]

(FYI I have this scheduled for review Thu Dec 20 at 13:00 UTC-05:00.)

[nikomatsakis]

OK, so I read this PR and the other one. @Zoxc let me summarize what I think is going on in the current PRs. I'm putting the comment here because I want to keep conversation "consolidated".

I believe that the current design basically has each rayon thread acquire a token from the jobserve before it starts looking for work and release that token when it goes to sleep, right?

This obviously makes a lot of sense, though I'm wondering a bit if there is some interaction with the LLVM compilation threads we want to be careful of. In particular, I believe that LLVM execution and (e.g.) trans can overlap -- this was a requirement to help us reduce peak memory usage requirements of incremental compilation, if I recall. Maybe we want to move those LLVM things into spawned rayon tasks, so that they are sharing the same basic thread-pool? (I've sort of forgotten how that system is setup, I'll have to investigate.)

I would definitely prefer if the rayon core code was "agnostic" as to the specifics of the thread-pool, as @michaelwoerister suggested. Given how simple the interface is, it basically seems like we are talking about adding two callbacks -- acquire_token and release_token -- to the rayon threadpool interface, right?

(The PR has some other changes, e.g., adopting #[thread_local], but I am not sure what the motivation for those changes is.)

[michaelwoerister]

Maybe we want to move those LLVM things into spawned rayon tasks, so that they are sharing the same basic thread-pool?

That's basically how I imagined this work in the future. The current LLVM scheduling is rather complicated but only because codegen/trans is bound to the main thread. All of this should get a lot simpler once the tcx can be shared between threads.

[Dylan-DPC-zz]

ping from triage @Zoxc@nikomatsakis any updates on this?

[Zoxc]

I've updated this to use the callbacks I added to Rayon. The jobserver module is now moved to rustc_data_structures (aka. rustc_misc).

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:01bf40e0:start=1548179329381587011,finish=1548179451168497109,duration=121786910098
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
############################################################ 84.7%
###################################################################### 98.0%
######################################################################## 100.0%
[00:01:30] extracting /checkout/obj/build/cache/2019-01-04/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:01:30] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:01:30] Caused by:
[00:01:30] No such file or directory (os error 2)
[00:01:30] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:01:30] Build completed unsuccessfully in 0:00:14
[00:01:30] Build completed unsuccessfully in 0:00:14
[00:01:30] Makefile:71: recipe for target 'prepare' failed
[00:01:30] make: *** [prepare] Error 1
[00:01:31] Command failed. Attempt 2/5:
[00:01:31] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:01:31] Caused by:
[00:01:31] No such file or directory (os error 2)
[00:01:31] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:01:31] Build completed unsuccessfully in 0:00:00
[00:01:31] Build completed unsuccessfully in 0:00:00
[00:01:31] Makefile:71: recipe for target 'prepare' failed
[00:01:31] make: *** [prepare] Error 1
[00:01:33] Command failed. Attempt 3/5:
[00:01:33] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:01:33] Caused by:
[00:01:33] No such file or directory (os error 2)
[00:01:33] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:01:33] Build completed unsuccessfully in 0:00:00
[00:01:33] Build completed unsuccessfully in 0:00:00
[00:01:33] make: *** [prepare] Error 1
[00:01:33] Makefile:71: recipe for target 'prepare' failed
[00:01:36] Command failed. Attempt 4/5:
[00:01:36] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:01:36] Caused by:
[00:01:36] No such file or directory (os error 2)
[00:01:36] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:01:36] Build completed unsuccessfully in 0:00:00
[00:01:36] Build completed unsuccessfully in 0:00:00
[00:01:36] make: *** [prepare] Error 1
[00:01:36] Makefile:71: recipe for target 'prepare' failed
[00:01:40] Command failed. Attempt 5/5:
[00:01:40] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:01:40] Caused by:
[00:01:40] No such file or directory (os error 2)
[00:01:40] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:01:40] Build completed unsuccessfully in 0:00:00

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Zoxc]

I'm wondering a bit if there is some interaction with the LLVM compilation threads we want to be careful of.

Rayon threads and LLVM threads will compete for jobserver tokens. I do want to get rid of the LLVM threads once parallel queries is on by default though.

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:033a3257:start=1548687577477865491,finish=1548687578611090201,duration=1133224710
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
[00:03:02] downloading https://static.rust-lang.org/dist/2019-01-18/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:03:02] ######################################################################## 100.0%
[00:03:03] extracting /checkout/obj/build/cache/2019-01-18/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:03:03] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:03:03] Caused by:
[00:03:03] No such file or directory (os error 2)
[00:03:03] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:03:03] Build completed unsuccessfully in 0:00:20
[00:03:03] Build completed unsuccessfully in 0:00:20
[00:03:03] make: *** [prepare] Error 1
[00:03:03] Makefile:70: recipe for target 'prepare' failed
[00:03:04] Command failed. Attempt 2/5:
[00:03:04] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:03:04] Caused by:
[00:03:04] No such file or directory (os error 2)
[00:03:04] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:03:04] Build completed unsuccessfully in 0:00:00
[00:03:04] Build completed unsuccessfully in 0:00:00
[00:03:04] Makefile:70: recipe for target 'prepare' failed
[00:03:04] make: *** [prepare] Error 1
[00:03:06] Command failed. Attempt 3/5:
[00:03:06] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:03:06] Caused by:
[00:03:06] No such file or directory (os error 2)
[00:03:06] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:03:06] Build completed unsuccessfully in 0:00:00
[00:03:06] Build completed unsuccessfully in 0:00:00
[00:03:06] make: *** [prepare] Error 1
[00:03:06] Makefile:70: recipe for target 'prepare' failed
[00:03:09] Command failed. Attempt 4/5:
[00:03:09] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:03:09] Caused by:
[00:03:09] No such file or directory (os error 2)
[00:03:09] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:03:09] Build completed unsuccessfully in 0:00:00
[00:03:09] Build completed unsuccessfully in 0:00:00
[00:03:09] make: *** [prepare] Error 1
[00:03:09] Makefile:70: recipe for target 'prepare' failed
[00:03:13] Command failed. Attempt 5/5:
[00:03:13] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:03:13] Caused by:
[00:03:13] No such file or directory (os error 2)
[00:03:13] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:03:13] Build completed unsuccessfully in 0:00:00
---
travis_time:end:115607eb:start=1548687786851108165,finish=1548687786856004355,duration=4896190
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0efa67b0
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0d089c00
travis_time:start:0d089c00
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:085dc3f4
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Zoxc]

Ping @nikomatsakis

[bors]

☔ The latest upstream changes (presumably #57948) made this pull request unmergeable. Please resolve the merge conflicts.

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:02bfb4c6:start=1548793653564940869,finish=1548793657726945160,duration=4162004291
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
[00:06:40] downloading https://static.rust-lang.org/dist/2019-01-18/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:06:41] ######################################################################## 100.0%
[00:06:41] extracting /checkout/obj/build/cache/2019-01-18/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:06:41] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:06:41] Caused by:
[00:06:41] No such file or directory (os error 2)
[00:06:41] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:06:41] Build completed unsuccessfully in 0:01:01
[00:06:41] Build completed unsuccessfully in 0:01:01
[00:06:41] make: *** [prepare] Error 1
[00:06:41] Makefile:70: recipe for target 'prepare' failed
[00:06:42] Command failed. Attempt 2/5:
[00:06:42] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:06:42] Caused by:
[00:06:42] No such file or directory (os error 2)
[00:06:42] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:06:42] Build completed unsuccessfully in 0:00:00
[00:06:42] Build completed unsuccessfully in 0:00:00
[00:06:42] make: *** [prepare] Error 1
[00:06:42] Makefile:70: recipe for target 'prepare' failed
[00:06:44] Command failed. Attempt 3/5:
[00:06:44] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:06:44] Caused by:
[00:06:44] No such file or directory (os error 2)
[00:06:44] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:06:44] Build completed unsuccessfully in 0:00:00
[00:06:44] Build completed unsuccessfully in 0:00:00
[00:06:44] Makefile:70: recipe for target 'prepare' failed
[00:06:44] make: *** [prepare] Error 1
[00:06:47] Command failed. Attempt 4/5:
[00:06:47] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:06:47] Caused by:
[00:06:47] No such file or directory (os error 2)
[00:06:47] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:06:47] Build completed unsuccessfully in 0:00:00
[00:06:47] Build completed unsuccessfully in 0:00:00
[00:06:47] Makefile:70: recipe for target 'prepare' failed
[00:06:47] make: *** [prepare] Error 1
[00:06:51] Command failed. Attempt 5/5:
[00:06:51] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:06:51] Caused by:
[00:06:51] No such file or directory (os error 2)
[00:06:51] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:06:51] Build completed unsuccessfully in 0:00:00
---
travis_time:end:0703faa5:start=1548794081949525081,finish=1548794081956056200,duration=6531119
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1ecfc00f
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:151583d9
travis_time:start:151583d9
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:014fd860
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[nikomatsakis]

Seems good, but I have a few questions and nits. =)

[nikomatsakis]

Why is this struct pub?

[nikomatsakis]

Why is this struct pub?

[nikomatsakis]

Maybe a doc-comment? (Also, does this struct need to be pub?)

[nikomatsakis]

Similarly, I believe these can be private to the module

[nikomatsakis]

This unsafe keyword feels problematic. In particular, it is asserting that GLOBAL_CLIENT is used early in the process, but it (locally, at least) has no way to know that. Basically, the clients of this module must invoke one of the public methods "early enough", right?

I feel like pub fn client should be declared unsafe, and parts of this comment moved onto it.

[Zoxc]

I don't think any of this is related to memory safety though.

[nikomatsakis]

There is no ordering dependency on when GLOBAL_PROXY is created vs GLOBAL_CLIENT, right?

[Zoxc]

GLOBAL_PROXY requires a GLOBAL_CLIENT.

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:213d55f1:start=1548950613012460478,finish=1548950614004998827,duration=992538349
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
[00:02:03] ############################################## 64.6%
######################################################################## 100.0%
[00:02:03] extracting /checkout/obj/build/cache/2019-01-18/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:02:03] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:02:03] Caused by:
[00:02:03] No such file or directory (os error 2)
[00:02:03] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:02:03] Build completed unsuccessfully in 0:00:13
[00:02:03] Build completed unsuccessfully in 0:00:13
[00:02:03] make: *** [prepare] Error 1
[00:02:03] Makefile:70: recipe for target 'prepare' failed
[00:02:04] Command failed. Attempt 2/5:
[00:02:04] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:02:04] Caused by:
[00:02:04] No such file or directory (os error 2)
[00:02:04] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:02:04] Build completed unsuccessfully in 0:00:00
[00:02:04] Build completed unsuccessfully in 0:00:00
[00:02:04] Makefile:70: recipe for target 'prepare' failed
[00:02:04] make: *** [prepare] Error 1
[00:02:06] Command failed. Attempt 3/5:
[00:02:06] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:02:06] Caused by:
[00:02:06] No such file or directory (os error 2)
[00:02:06] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:02:06] Build completed unsuccessfully in 0:00:00
[00:02:06] Build completed unsuccessfully in 0:00:00
[00:02:06] Makefile:70: recipe for target 'prepare' failed
[00:02:06] make: *** [prepare] Error 1
[00:02:09] Command failed. Attempt 4/5:
[00:02:10] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:02:10] Caused by:
[00:02:10] No such file or directory (os error 2)
[00:02:10] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:02:10] Build completed unsuccessfully in 0:00:00
[00:02:10] Build completed unsuccessfully in 0:00:00
[00:02:10] make: *** [prepare] Error 1
[00:02:10] Makefile:70: recipe for target 'prepare' failed
[00:02:14] Command failed. Attempt 5/5:
[00:02:14] error: failed to read `/par/rayon-tlv/Cargo.toml`
[00:02:14] Caused by:
[00:02:14] No such file or directory (os error 2)
[00:02:14] failed to run: /checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /checkout/src/bootstrap/Cargo.toml --locked
[00:02:14] Build completed unsuccessfully in 0:00:00
---
travis_time:end:26b7ab04:start=1548950762198461327,finish=1548950762203164341,duration=4703014
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:13727568
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:1723865a
travis_time:start:1723865a
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:06c07b54
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Zoxc]

@bors r=nikomatsakis

[bors]

📌 Commit a21370f8978ab051289de2bcc6f29fa2c81584af has been approved by nikomatsakis

[bors]

☔ The latest upstream changes (presumably #58250) made this pull request unmergeable. Please resolve the merge conflicts.

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:0de9d599:start=1551399466760896147,finish=1551399468895162958,duration=2134266811
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
$ pip install --user awscli; export PATH=$PATH:$HOME/.local/bin:$HOME/Library/Python/2.7/bin/
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
warnings.warn(warning, RequestsDependencyWarning)
Collecting awscli
Downloading https://files.pythonhosted.org/packages/aa/ea/cb62728e9b38f9d8c620d60815f8dd54ca015f6b9af8f5a3d03d9b2e3c64/awscli-1.16.115-py2.py3-none-any.whl (1.4MB)
1% |▌ | 20kB 2.1MB/s eta 0:00:01
2% |▊ | 30kB 3.0MB/s eta 0:00:01
2% |█ | 40kB 2.0MB/s eta 0:00:01
3% |█▏ | 51kB 2.5MB/s eta 0:00:01
---
99% |████████████████████████████████| 542kB 47.4MB/s eta 0:00:01
100% |████████████████████████████████| 552kB 25.3MB/s Requirement already satisfied: PyYAML<=3.13,>=3.10 in /usr/lib/python2.7/dist-packages (from awscli) (3.11)
Collecting botocore==1.12.105 (from awscli)
Downloading https://files.pythonhosted.org/packages/cf/ce/acc9013dee20fc94c9b9ae121f5b7b342a206f0d577be1e5c6129811194a/botocore-1.12.105-py2.py3-none-any.whl (5.3MB)
0% |▏ | 20kB 18.9MB/s eta 0:00:01
0% |▏ | 30kB 23.0MB/s eta 0:00:01
0% |▎ | 40kB 23.1MB/s eta 0:00:01
0% |▎ | 51kB 19.1MB/s eta 0:00:01
---
########################################################### 83.3%
######################################################################## 100.0%
[00:02:13] extracting /checkout/obj/build/cache/2019-02-17/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:02:13] Updating crates.io index
[00:02:32] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:32] Build completed unsuccessfully in 0:00:40
[00:02:32] make: *** [prepare] Error 1
[00:02:32] Makefile:70: recipe for target 'prepare' failed
[00:02:33] Command failed. Attempt 2/5:
[00:02:33] Command failed. Attempt 2/5:
[00:02:34] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:34] Build completed unsuccessfully in 0:00:00
[00:02:34] make: *** [prepare] Error 1
[00:02:34] Makefile:70: recipe for target 'prepare' failed
[00:02:36] Command failed. Attempt 3/5:
[00:02:36] Command failed. Attempt 3/5:
[00:02:36] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:36] Build completed unsuccessfully in 0:00:00
[00:02:36] make: *** [prepare] Error 1
[00:02:36] Makefile:70: recipe for target 'prepare' failed
[00:02:39] Command failed. Attempt 4/5:
[00:02:39] Command failed. Attempt 4/5:
[00:02:39] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:39] Build completed unsuccessfully in 0:00:00
[00:02:39] make: *** [prepare] Error 1
[00:02:39] Makefile:70: recipe for target 'prepare' failed
[00:02:43] Command failed. Attempt 5/5:
[00:02:43] Command failed. Attempt 5/5:
[00:02:44] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:02:44] Build completed unsuccessfully in 0:00:00
[00:02:44] Makefile:70: recipe for target 'prepare' failed
[00:02:44] The command has failed after 5 attempts.
[00:02:44] make: *** [prepare] Error 1

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Zoxc]

@bors r=nikomatsakis

[bors]

📌 Commit 5c78fa8 has been approved by nikomatsakis

[bors]

⌛ Testing commit 5c78fa8 with merge d987b46...

[bors]

☀️ Test successful - checks-travis, status-appveyor
Approved by: nikomatsakis
Pushing d987b46 to master...

[michaelwoerister]

It would be great if we could have some kind of regression test for this. I don't know how to do that though (we don't have tests for the other jobserver support either).