rustc: implement argsfiles for command line by jsgf · Pull Request #63175 · rust-lang/rust

jsgf · 2019-07-31T21:44:07Z

Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line. This causes file to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \-escaped. Args files may recursively include other args files via @file2.

See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters.

This is useful for very large command lines, or when command lines are being generated into files by other tooling.

rust-highfive · 2019-07-31T21:44:10Z

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2019-07-31T21:56:13Z

I clearly remember I've seen an issue about this, but I can't find it right now.

MSVC supports them as well, with the same syntax.

Mark-Simulacrum · 2019-07-31T22:39:13Z

cc @alexcrichton since I believe you added our support for calling linkers with this and are probably generally interested due to Cargo concerns

Mark-Simulacrum · 2019-07-31T22:42:45Z

It feels a bit error-prone to simply ignore @files that are "too deep". Maybe we can at least eprintln or something here? There's probably no good way to give a good error message here...
One possible "solution" is to just panic and bump MAX_RECURSION to some very large number like 1000 which would be relatively unreasonable.

The documented error handling is pretty awkward - it also just includes the literal @file string as an arg if it can't open/read the file. The recursion handling is at least consistent with that low standard.
I'm happy to scatter some eprintlns (if that's the best that's available here) around to make noise about things going wrong.

Mark-Simulacrum · 2019-07-31T22:45:25Z

Though this matches the ld spec, it is worth noting that this is the first place in rustc/cargo/etc where we're splitting by whitespace and implementing quoting rules around that. I haven't reviewed the implementation here closely but maybe we could avoid doing that somehow?

petrochenkov · 2019-07-31T22:45:39Z

r? @alexcrichton

cramertj · 2019-07-31T22:53:09Z

Hey, I was just wanting this! Wonderful :)

jsgf · 2019-07-31T23:09:34Z

@Mark-Simulacrum I was following this as a convention rather than novelty for the sake of it. I chose gnu-ld as a model because it was the first even slightly detailed description of the file syntax which made sense (tho there's still a fair amount of ambiguity).

I could go with something simpler like "each arg is on its own line", though that raises questions like:

What if an arg contains a newline? Doesn't seem completely unreasonable.
How to define newlines on Windows? Insist on full \r\n, or also handle Unix \n line breaks? What does \r on its own mean?

The this implementation:

consumes leading and trailing whitespace in an unsurprising way
allows for empty args by quoting ""
handles space, tab, \r and \n uniformly as whitespace which eliminates line-ending ambiguity
Handles '...' and "..." in a shell-like way, as well as \ quoting.

This has some amount of superfluous complexity, but I think that's offset by making it compatible with typical shell quoting rules (so, for example, Python's shlex module can be used to quote args in this form - with some care).

Mark-Simulacrum · 2019-07-31T23:20:40Z

I completely agree that this is a viable and maybe best implementation; just wanted to raise the concern that we've elsewhere always tried to shy away from shlex or the like.

One solution might be to use a null byte as a separator, since iirc they're not valid utf-8 (and rustc generally doesn't work with anything requiring them, anyway). Since these files are presumably mostly programmatically generated that doesn't seem too bad.

jsgf · 2019-07-31T23:25:09Z

@Mark-Simulacrum \0-separated is an option, but its a bit painful to generate from a shell script and would make the files be treated as binary and unmergable by a source control system (eg, tests in rustc itself).

jsgf · 2019-08-01T00:03:22Z

BTW, where does $DIR expansion happen? It doesn't seem to be happening for @$DIR/.... Does the $ need to be the first non-whitespace character?

Mark-Simulacrum · 2019-08-01T00:32:23Z

$DIR among other expansions happens in normalize_output.

I think my personal preference is \0 as the separator over implementing shlex-alikes but I don't care too much.

jsgf · 2019-08-01T01:22:24Z

@Mark-Simulacrum Ah thanks, I had thought that it could be used in the .rs file as an expansion. Test fixed (I hope).

jsgf · 2019-08-20T02:08:43Z

OK:

moved the named args for usage to be first in the stack
folded all previous work into a single commit
fixed up error message normalization for windows and addressed other comments
added commit to move argfile expansion to run_compiler

rust-highfive · 2019-08-20T02:26:02Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-20T02:07:38.2708746Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-20T02:07:38.2889638Z ##[command]git config gc.auto 0
2019-08-20T02:07:38.2956615Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-20T02:07:38.3007332Z ##[command]git config --get-all http.proxy
2019-08-20T02:07:38.3129869Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-20T02:08:12.4801322Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-20T02:08:12.4801369Z 2019-08-20T02:08:12.4801543Z git checkout -b <new-branch-name>
2019-08-20T02:08:12.4801568Z 2019-08-20T02:08:12.4801609Z HEAD is now at c3148b5e0 Merge 05e845ee966945870722ac4100baca5c58f01d9d into c1b08dd26036e14f061b99b20cd6f169e29046f3
2019-08-20T02:08:12.4951186Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-20T02:08:12.4953985Z ==============================================================================
2019-08-20T02:08:12.4954034Z Task : Bash
2019-08-20T02:08:12.4954088Z Description : Run a Bash script on macOS, Linux, or Windows
---
2019-08-20T02:14:27.5664047Z Compiling serde_json v1.0.40
2019-08-20T02:14:29.3950943Z Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2019-08-20T02:14:40.0575161Z Finished release [optimized] target(s) in 1m 28s
2019-08-20T02:14:40.0660676Z tidy check
2019-08-20T02:14:40.5836707Z tidy error: /checkout/src/librustc_driver/lib.rs:152: line longer than 100 chars
2019-08-20T02:14:40.8681487Z tidy error: /checkout/src/test/ui/commandline-argfile-missing.rs:5: line longer than 100 chars
2019-08-20T02:14:42.0269063Z some tidy checks failed
2019-08-20T02:14:42.0269247Z 2019-08-20T02:14:42.0269247Z 2019-08-20T02:14:42.0270047Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2019-08-20T02:14:42.0270205Z 2019-08-20T02:14:42.0270228Z 2019-08-20T02:14:42.0270271Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2019-08-20T02:14:42.0270317Z Build completed unsuccessfully in 0:01:31
2019-08-20T02:14:42.0270317Z Build completed unsuccessfully in 0:01:31
2019-08-20T02:14:42.0313654Z == clock drift check ==
2019-08-20T02:14:42.0331134Z local time: Tue Aug 20 02:14:42 UTC 2019
2019-08-20T02:14:42.1847520Z network time: Tue, 20 Aug 2019 02:14:42 GMT
2019-08-20T02:14:42.1848233Z == end clock drift check ==
2019-08-20T02:14:43.6071817Z ##[error]Bash exited with code '1'.
2019-08-20T02:14:43.6103766Z ##[section]Starting: Checkout
2019-08-20T02:14:43.6105879Z ==============================================================================
2019-08-20T02:14:43.6105939Z Task : Get sources
2019-08-20T02:14:43.6106007Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-20T03:53:18Z

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-20T03:23:53.8868432Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-20T03:23:53.9090586Z ##[command]git config gc.auto 0
2019-08-20T03:23:53.9156280Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-20T03:23:53.9210440Z ##[command]git config --get-all http.proxy
2019-08-20T03:23:53.9352662Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-20T03:24:28.3731432Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-20T03:24:28.3731699Z 2019-08-20T03:24:28.3732123Z git checkout -b <new-branch-name>
2019-08-20T03:24:28.3732345Z 2019-08-20T03:24:28.3732540Z HEAD is now at c6dc7feb0 Merge b409b652026a17efa565c493b491b98b341bff14 into c1b08dd26036e14f061b99b20cd6f169e29046f3
2019-08-20T03:24:28.3906418Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-20T03:24:28.3909265Z ==============================================================================
2019-08-20T03:24:28.3909321Z Task : Bash
2019-08-20T03:24:28.3909383Z Description : Run a Bash script on macOS, Linux, or Windows
---
2019-08-20T03:33:35.9604151Z Checking rustc_interface v0.0.0 (/checkout/src/librustc_interface)
2019-08-20T03:33:38.6037137Z error[E0308]: mismatched types
2019-08-20T03:33:38.6040008Z --> src/librustc_driver/lib.rs:1216:22
2019-08-20T03:33:38.6040723Z |
2019-08-20T03:33:38.6041306Z 1216 | run_compiler(args, &mut callbacks, None, None)
2019-08-20T03:33:38.6043053Z | |
2019-08-20T03:33:38.6044424Z | expected reference, found struct `std::vec::Vec`
2019-08-20T03:33:38.6044424Z | expected reference, found struct `std::vec::Vec`
2019-08-20T03:33:38.6046236Z | help: consider borrowing here: `&args`
2019-08-20T03:33:38.6046845Z |
2019-08-20T03:33:38.6047518Z = note: expected type `&[std::string::String]`
2019-08-20T03:33:38.6048298Z found type `std::vec::Vec<std::string::String>`
2019-08-20T03:33:38.6963703Z error: aborting due to previous error
2019-08-20T03:33:38.6963947Z 2019-08-20T03:33:38.6965073Z For more information about this error, try `rustc --explain E0308`.
2019-08-20T03:33:38.7173787Z error: Could not compile `rustc_driver`.
---
2019-08-20T03:33:38.7271516Z == clock drift check ==
2019-08-20T03:33:38.7288112Z local time: Tue Aug 20 03:33:38 UTC 2019
2019-08-20T03:33:38.8142321Z network time: Tue, 20 Aug 2019 03:33:38 GMT
2019-08-20T03:33:38.8147364Z == end clock drift check ==
2019-08-20T03:33:40.1099488Z ##[error]Bash exited with code '1'.
2019-08-20T03:33:40.1140431Z ##[section]Starting: Checkout
2019-08-20T03:33:40.1141923Z ==============================================================================
2019-08-20T03:33:40.1141972Z Task : Get sources
2019-08-20T03:33:40.1142015Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-20T06:17:46Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-20T05:03:58.1101205Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-20T05:03:58.1326856Z ##[command]git config gc.auto 0
2019-08-20T05:03:58.1412341Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-20T05:03:58.1463165Z ##[command]git config --get-all http.proxy
2019-08-20T05:03:58.1589690Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-20T05:04:33.1629252Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-20T05:04:33.1629285Z 2019-08-20T05:04:33.1629523Z git checkout -b <new-branch-name>
2019-08-20T05:04:33.1629571Z 2019-08-20T05:04:33.1629625Z HEAD is now at b0dc87b26 Merge 3f58528665cbd8b31944e91b7fbe1aa8213f95a2 into d8d99baf565bb389b82553dfe1f6e56ddc650184
2019-08-20T05:04:33.1787668Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-20T05:04:33.1790461Z ==============================================================================
2019-08-20T05:04:33.1790520Z Task : Bash
2019-08-20T05:04:33.1790568Z Description : Run a Bash script on macOS, Linux, or Windows
---
2019-08-20T06:06:33.2035333Z .................................................................................................... 1500/8944
2019-08-20T06:06:38.7235932Z .................................................................................................... 1600/8944
2019-08-20T06:06:51.6170635Z ........................................i...............i........................................... 1700/8944
2019-08-20T06:06:59.6227267Z .................................................................................................... 1800/8944
2019-08-20T06:07:14.0275866Z ................................iiiii............................................................... 1900/8944
2019-08-20T06:07:24.6660293Z .................................................................................................... 2100/8944
2019-08-20T06:07:27.2455276Z .................................................................................................... 2200/8944
2019-08-20T06:07:31.8949093Z .................................................................................................... 2300/8944
2019-08-20T06:07:38.9176490Z .................................................................................................... 2400/8944
---
2019-08-20T06:10:40.8690916Z ....................i...............i............................................................... 4700/8944
2019-08-20T06:10:52.8843027Z .................................................................................................... 4800/8944
2019-08-20T06:10:59.1910571Z .................................................................................................... 4900/8944
2019-08-20T06:11:10.4603880Z .................................................................................................... 5000/8944
2019-08-20T06:11:15.8631320Z .ii.ii.............................................................................................. 5100/8944
2019-08-20T06:11:30.9268436Z .................................................................................................... 5300/8944
2019-08-20T06:11:38.0554698Z .........................................................i.......................................... 5400/8944
2019-08-20T06:11:45.2773319Z .................................................................................................... 5500/8944
2019-08-20T06:11:54.2301630Z .................................................................................................... 5600/8944
2019-08-20T06:11:54.2301630Z .................................................................................................... 5600/8944
2019-08-20T06:12:04.5947612Z ..................................................ii...i..ii...........i............................ 5700/8944
2019-08-20T06:12:27.0140698Z .................................................................................................... 5900/8944
2019-08-20T06:12:32.1281603Z .................................................................................................... 6000/8944
2019-08-20T06:12:32.1281603Z .................................................................................................... 6000/8944
2019-08-20T06:12:42.8104644Z ...................................................i..ii............................................ 6100/8944
2019-08-20T06:13:07.1331620Z .................................................................................................i.. 6300/8944
2019-08-20T06:13:09.4004620Z .................................................................................................... 6400/8944
2019-08-20T06:13:11.6840223Z .....................................................................i.............................. 6500/8944
2019-08-20T06:13:14.7015826Z .................................................................................................... 6600/8944
---
2019-08-20T06:17:19.0152086Z 2019-08-20T06:17:19.0152854Z ---- [ui] ui/commandline-argfile-missing.rs stdout ----
2019-08-20T06:17:19.0153181Z diff of stderr:
2019-08-20T06:17:19.0153440Z 2019-08-20T06:17:19.0154012Z - error: Failed to load argument file: IO Error: $DIR/commandline-argfile-missing.args: $FILE_MISSING (os error $ERR)
2019-08-20T06:17:19.0154665Z + error: Failed to load argument file: IO Error: $DIR/commandline-argfile-missing.args: No such file or directory (os error $ERR)
2019-08-20T06:17:19.0155451Z 3 2019-08-20T06:17:19.0155672Z 2019-08-20T06:17:19.0155876Z 2019-08-20T06:17:19.0156109Z The actual stderr differed from the expected stderr.
2019-08-20T06:17:19.0156109Z The actual stderr differed from the expected stderr.
2019-08-20T06:17:19.0156712Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile-missing/commandline-argfile-missing.stderr
2019-08-20T06:17:19.0157295Z To update references, rerun the tests and pass the `--bless` flag
2019-08-20T06:17:19.0157880Z To only update this specific test, also pass `--test-args commandline-argfile-missing.rs`
2019-08-20T06:17:19.0158391Z error: 1 errors occurred comparing output.
2019-08-20T06:17:19.0158615Z status: exit code: 1
2019-08-20T06:17:19.0158615Z status: exit code: 1
2019-08-20T06:17:19.0159979Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/commandline-argfile-missing.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile-missing" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--cfg" "cmdline_set" "@/checkout/src/test/ui/commandline-argfile-missing.args" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/commandline-argfile-missing/auxiliary" "-A" "unused"
2019-08-20T06:17:19.0160958Z ------------------------------------------
2019-08-20T06:17:19.0161528Z 2019-08-20T06:17:19.0162069Z ------------------------------------------
2019-08-20T06:17:19.0162391Z stderr:
2019-08-20T06:17:19.0162391Z stderr:
2019-08-20T06:17:19.0162854Z ------------------------------------------
2019-08-20T06:17:19.0163480Z error: Failed to load argument file: IO Error: /checkout/src/test/ui/commandline-argfile-missing.args: No such file or directory (os error 2)
2019-08-20T06:17:19.0163962Z 2019-08-20T06:17:19.0164447Z ------------------------------------------
2019-08-20T06:17:19.0164708Z 2019-08-20T06:17:19.0164922Z ---
2019-08-20T06:17:19.0187710Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-08-20T06:17:19.0188087Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-20T06:17:19.0201861Z 2019-08-20T06:17:19.0201930Z 2019-08-20T06:17:19.0213266Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-20T06:17:19.0213687Z 2019-08-20T06:17:19.0214478Z 2019-08-20T06:17:19.0217002Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-20T06:17:19.0217222Z Build completed unsuccessfully in 1:06:13
2019-08-20T06:17:19.0217222Z Build completed unsuccessfully in 1:06:13
2019-08-20T06:17:19.0271045Z == clock drift check ==
2019-08-20T06:17:19.0288319Z local time: Tue Aug 20 06:17:19 UTC 2019
2019-08-20T06:17:19.1656338Z network time: Tue, 20 Aug 2019 06:17:19 GMT
2019-08-20T06:17:19.1659223Z == end clock drift check ==
2019-08-20T06:17:20.0203858Z ##[error]Bash exited with code '1'.
2019-08-20T06:17:20.0246333Z ##[section]Starting: Checkout
2019-08-20T06:17:20.0248203Z ==============================================================================
2019-08-20T06:17:20.0248279Z Task : Get sources
2019-08-20T06:17:20.0248331Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

alexcrichton · 2019-08-20T14:40:13Z

@bors: delegate+

Implementation looks good to me, feel free to r+ when tests are good

bors · 2019-08-20T14:40:14Z

✌️ @jsgf can now approve this pull request

rust-highfive · 2019-08-20T19:32:07Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-20T17:47:51.5190858Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-20T17:47:51.5426086Z ##[command]git config gc.auto 0
2019-08-20T17:47:51.5492033Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-20T17:47:51.5556467Z ##[command]git config --get-all http.proxy
2019-08-20T17:47:51.5700960Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63175/merge:refs/remotes/pull/63175/merge
---
2019-08-20T17:48:27.2987858Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-20T17:48:27.2988658Z 2019-08-20T17:48:27.2989214Z git checkout -b <new-branch-name>
2019-08-20T17:48:27.2989468Z 2019-08-20T17:48:27.2989775Z HEAD is now at 6c18e4db6 Merge 44753eb57fa79ea6551631d3213f416edec96a64 into 51879c3abaedb926739095d19a2af638ee6a07d8
2019-08-20T17:48:27.3142618Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-20T17:48:27.3145441Z ==============================================================================
2019-08-20T17:48:27.3145539Z Task : Bash
2019-08-20T17:48:27.3145586Z Description : Run a Bash script on macOS, Linux, or Windows
---
2019-08-20T18:52:22.6167931Z .................................................................................................... 1500/8946
2019-08-20T18:52:28.4312602Z .................................................................................................... 1600/8946
2019-08-20T18:52:41.6173423Z ...........................................i..............i......................................... 1700/8946
2019-08-20T18:52:50.1213304Z .................................................................................................... 1800/8946
2019-08-20T18:53:04.8962476Z ..................................iiiii............................................................. 1900/8946
2019-08-20T18:53:15.9803740Z .................................................................................................... 2100/8946
2019-08-20T18:53:18.6876034Z .................................................................................................... 2200/8946
2019-08-20T18:53:23.4005744Z .................................................................................................... 2300/8946
2019-08-20T18:53:30.7817108Z .................................................................................................... 2400/8946
---
2019-08-20T18:56:36.7410689Z ......................i...............i............................................................. 4700/8946
2019-08-20T18:56:48.8107888Z .................................................................................................... 4800/8946
2019-08-20T18:56:55.1567007Z .................................................................................................... 4900/8946
2019-08-20T18:57:06.5939656Z .................................................................................................... 5000/8946
2019-08-20T18:57:12.1346348Z ...ii.ii............................................................................................ 5100/8946
2019-08-20T18:57:27.4528885Z .................................................................................................... 5300/8946
2019-08-20T18:57:34.4970373Z ...........................................................i........................................ 5400/8946
2019-08-20T18:57:41.8624854Z .................................................................................................... 5500/8946
2019-08-20T18:57:50.5032955Z .................................................................................................... 5600/8946
2019-08-20T18:57:50.5032955Z .................................................................................................... 5600/8946
2019-08-20T18:58:01.3347447Z ....................................................ii...i..ii...........i.......................... 5700/8946
2019-08-20T18:58:25.4024974Z .................................................................................................... 5900/8946
2019-08-20T18:58:30.5880612Z .................................................................................................... 6000/8946
2019-08-20T18:58:30.5880612Z .................................................................................................... 6000/8946
2019-08-20T18:58:38.5929304Z .....................................................i..ii.......................................... 6100/8946
2019-08-20T18:59:06.4461003Z ...................................................................................................i 6300/8946
2019-08-20T18:59:08.7848895Z .................................................................................................... 6400/8946
2019-08-20T18:59:11.1439353Z .......................................................................i............................ 6500/8946
2019-08-20T18:59:14.3166663Z .................................................................................................... 6600/8946
---
2019-08-20T19:04:05.0920848Z finished in 21.788
2019-08-20T19:04:05.1127947Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:04:05.2790280Z 2019-08-20T19:04:05.2791005Z running 148 tests
2019-08-20T19:04:08.6898988Z i....iii......iii..iiii....i............................i..i..................i....i.........ii.i.i. 100/148
2019-08-20T19:04:10.6581216Z .iiii..............i.........iii.i......ii......
2019-08-20T19:04:10.6582255Z 2019-08-20T19:04:10.6583910Z finished in 5.545
2019-08-20T19:04:10.6793512Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:04:10.8480289Z ---
2019-08-20T19:04:12.9914658Z finished in 2.312
2019-08-20T19:04:13.0143714Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:04:13.1870858Z 2019-08-20T19:04:13.1871089Z running 9 tests
2019-08-20T19:04:13.1871891Z iiiiiiiii
2019-08-20T19:04:13.1872301Z 2019-08-20T19:04:13.1872348Z finished in 0.173
2019-08-20T19:04:13.2067241Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:04:13.3735533Z ---
2019-08-20T19:04:32.0949888Z finished in 18.887
2019-08-20T19:04:32.1159280Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:04:32.2894190Z 2019-08-20T19:04:32.2894481Z running 122 tests
2019-08-20T19:04:57.6363169Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....i..........iiii..........i...ii...i.......ii.i 100/122
2019-08-20T19:05:02.5658207Z .i.i......iii.i.....ii
2019-08-20T19:05:02.5658679Z 2019-08-20T19:05:02.5662232Z finished in 30.450
2019-08-20T19:05:02.5669754Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:05:02.5670115Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-08-20T19:19:36.1053146Z running 970 tests
2019-08-20T19:19:36.1217393Z .................................................................................................... 100/970
2019-08-20T19:19:36.1322550Z .................................................................................................... 200/970
2019-08-20T19:19:36.1418081Z .................................................................................................... 300/970
2019-08-20T19:19:36.2328098Z ................................................................................................ii.. 400/970
2019-08-20T19:19:36.3509409Z .................................................................................................... 600/970
2019-08-20T19:19:36.3757809Z .................................................................................................... 700/970
2019-08-20T19:19:36.3870675Z .................................................................................................... 800/970
2019-08-20T19:19:36.9312814Z .................................................................................................... 900/970
2019-08-20T19:19:36.9312814Z .................................................................................................... 900/970
2019-08-20T19:19:38.1225310Z ......................................................................
2019-08-20T19:19:38.1231941Z test result: ok. 968 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out
2019-08-20T19:19:38.1232349Z 2019-08-20T19:19:38.1241024Z Doc-tests core
2019-08-20T19:19:43.5558545Z 2019-08-20T19:19:43.5566893Z running 2380 tests
2019-08-20T19:19:57.2493150Z ......iiiii......................................................................................... 100/2380
2019-08-20T19:20:11.1194321Z .........................................................................ii......................... 200/2380
2019-08-20T19:20:43.8391928Z .................................................................................................... 400/2380
2019-08-20T19:20:43.8391928Z .................................................................................................... 400/2380
2019-08-20T19:20:56.1957455Z ..............................i..i.................iiii............................................. 500/2380
2019-08-20T19:21:22.7430922Z .................................................................................................... 700/2380
2019-08-20T19:21:35.9339874Z .................................................................................................... 800/2380
2019-08-20T19:21:49.1624488Z .................................................................................................... 900/2380
2019-08-20T19:22:02.4776207Z .................................................................................................... 1000/2380
---
2019-08-20T19:27:53.0446596Z 2019-08-20T19:27:53.0447725Z running 991 tests
2019-08-20T19:28:17.9362255Z i................................................................................................... 100/991
2019-08-20T19:28:31.7987500Z .................................................................................................... 200/991
2019-08-20T19:28:41.1487525Z .................iii......i......i...i......i....................................................... 300/991
2019-08-20T19:28:46.5907169Z .................................................................................................... 400/991
2019-08-20T19:28:55.2114088Z ..................................i..i.................................ii........................... 500/991
2019-08-20T19:29:11.3850391Z .................................................................................................... 700/991
2019-08-20T19:29:11.3850391Z .................................................................................................... 700/991
2019-08-20T19:29:20.1938327Z .................iiii............................................................................... 800/991
2019-08-20T19:29:35.8802339Z .................................................................................................... 900/991
2019-08-20T19:29:44.0625022Z .......................................iiii................................................
2019-08-20T19:29:44.0625864Z 2019-08-20T19:29:44.0835838Z finished in 264.478
2019-08-20T19:29:44.0853338Z Testing unwind stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-20T19:29:44.2512366Z Finished release [optimized] target(s) in 0.16s
---
2019-08-20T19:31:37.3936862Z Compiling rustc_driver v0.0.0 (/checkout/src/librustc_driver)
2019-08-20T19:31:37.3937673Z error[E0583]: file not found for module `tests`
2019-08-20T19:31:37.3938117Z --> src/librustc_driver/args.rs:9:5
2019-08-20T19:31:37.3938489Z |
2019-08-20T19:31:37.3938889Z 9 | mod tests;
2019-08-20T19:31:37.3940120Z |
2019-08-20T19:31:37.3940120Z |
2019-08-20T19:31:37.3940640Z = help: name the file either args/tests.rs or args/tests/mod.rs inside the directory "src/librustc_driver"
2019-08-20T19:31:37.3941199Z error: aborting due to previous error
2019-08-20T19:31:37.3941380Z 2019-08-20T19:31:37.3942704Z For more information about this error, try `rustc --explain E0583`.
2019-08-20T19:31:37.3943245Z error: Could not compile `rustc_driver`.
2019-08-20T19:31:37.3943245Z error: Could not compile `rustc_driver`.
2019-08-20T19:31:37.3943438Z 2019-08-20T19:31:37.3943821Z To learn more, run the command again with --verbose.
2019-08-20T19:31:37.3943979Z 2019-08-20T19:31:37.3944104Z 2019-08-20T19:31:37.3945206Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_driver" "--" "--quiet"
2019-08-20T19:31:37.3946120Z 2019-08-20T19:31:37.3946255Z 2019-08-20T19:31:37.3946400Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-20T19:31:37.3946567Z Build completed unsuccessfully in 1:36:09
2019-08-20T19:31:37.3946567Z Build completed unsuccessfully in 1:36:09
2019-08-20T19:31:37.3946713Z == clock drift check ==
2019-08-20T19:31:37.3946868Z local time: Tue Aug 20 19:31:37 UTC 2019
2019-08-20T19:31:37.3947038Z network time: Tue, 20 Aug 2019 19:31:37 GMT
2019-08-20T19:31:37.3947180Z == end clock drift check ==
2019-08-20T19:31:37.8512557Z ##[error]Bash exited with code '1'.
2019-08-20T19:31:37.8556060Z ##[section]Starting: Checkout
2019-08-20T19:31:37.8557814Z ==============================================================================
2019-08-20T19:31:37.8557892Z Task : Get sources
2019-08-20T19:31:37.8557941Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

This will make @path work with miri and other non-standard entrypoints. Also since this simplifies librustc_driver::args, move it into a simple source file. Also remove the tests since they're doing nothing more than checking `str::lines` has the right behaviour.

jsgf · 2019-08-22T00:39:19Z

@bors r+

bors · 2019-08-22T00:39:20Z

📌 Commit d949774 has been approved by jsgf

bors · 2019-08-22T02:09:23Z

⌛ Testing commit d949774 with merge 54ebbd0...

rustc: implement argsfiles for command line Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line. This causes `file` to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \\-escaped. Args files may recursively include other args files via `@file2`. See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters. This is useful for very large command lines, or when command lines are being generated into files by other tooling.

bors · 2019-08-22T06:09:23Z

💥 Test timed out

jsgf · 2019-08-22T06:14:44Z

@bors retry

bors · 2019-08-22T06:14:58Z

⌛ Testing commit d949774 with merge 201e52e...

rustc: implement argsfiles for command line Many tools, such as gcc and gnu-ld, support "args files" - that is, being able to specify @file on the command line. This causes `file` to be opened and parsed for command line options. They're separated with whitespace; whitespace can be quoted with double or single quotes, and everything can be \\-escaped. Args files may recursively include other args files via `@file2`. See https://sourceware.org/binutils/docs/ld/Options.html#Options for the documentation of gnu-ld's @file parameters. This is useful for very large command lines, or when command lines are being generated into files by other tooling.

bors · 2019-08-22T10:02:03Z

☀️ Test successful - checks-azure
Approved by: jsgf
Pushing 201e52e to master...

rust-highfive assigned petrochenkov Jul 31, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 31, 2019

petrochenkov added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jul 31, 2019

This comment has been minimized.
Sign in to view

jsgf force-pushed the argsfile branch 2 times, most recently from cf1bc47 to fbcaacfCompare July 31, 2019 22:38

Centril added the relnotes Marks issues that should be documented in the release notes of the next release. label Jul 31, 2019

Mark-Simulacrum reviewed Jul 31, 2019
View reviewed changes

rust-highfive assigned alexcrichton and unassigned petrochenkov Jul 31, 2019

Centril added the needs-fcp This change is insta-stable, or significant enough to need a team FCP to proceed. label Jul 31, 2019

jsgf force-pushed the argsfile branch from fbcaacf to 9291a1cCompare July 31, 2019 23:21

jsgf force-pushed the argsfile branch from 9291a1c to 94f3d58Compare August 1, 2019 00:03

This comment has been minimized.
Sign in to view

jsgf force-pushed the argsfile branch from 94f3d58 to c46e0a4Compare August 1, 2019 01:21

This comment has been minimized.
Sign in to view

jsgf force-pushed the argsfile branch 2 times, most recently from 746623c to f3620d5Compare August 1, 2019 15:45

jsgf force-pushed the argsfile branch 2 times, most recently from 9f51b9e to b409b65Compare August 20, 2019 03:23

jsgf force-pushed the argsfile branch from b409b65 to 3f58528Compare August 20, 2019 05:03

jsgf force-pushed the argsfile branch 2 times, most recently from 07fc983 to 44753ebCompare August 20, 2019 17:47

jsgf force-pushed the argsfile branch from 44753eb to d949774Compare August 20, 2019 19:34

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 22, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 22, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 22, 2019

bors merged commit d949774 into rust-lang:masterAug 22, 2019

djkoloski mentioned this pull request Oct 26, 2023
Support response files generated by ninja with @ninja:path syntax rust-lang/compiler-team#684
Closed
3 tasks

Uh oh!

Uh oh!

Conversation

jsgf commented Jul 31, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented Jul 31, 2019

Uh oh!

petrochenkov commented Jul 31, 2019

Uh oh!

This comment has been minimized.

Mark-Simulacrum commented Jul 31, 2019

Uh oh!

Mark-SimulacrumJul 31, 2019

Choose a reason for hiding this comment

Uh oh!

jsgfJul 31, 2019

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum commented Jul 31, 2019

Uh oh!

petrochenkov commented Jul 31, 2019

Uh oh!

cramertj commented Jul 31, 2019

Uh oh!

jsgf commented Jul 31, 2019

Uh oh!

Mark-Simulacrum commented Jul 31, 2019

Uh oh!

jsgf commented Jul 31, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jsgf commented Aug 1, 2019

Uh oh!

Mark-Simulacrum commented Aug 1, 2019

Uh oh!

This comment has been minimized.

jsgf commented Aug 1, 2019

Uh oh!

This comment has been minimized.

jsgf commented Aug 20, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented Aug 20, 2019

Uh oh!

rust-highfive commented Aug 20, 2019

Uh oh!

rust-highfive commented Aug 20, 2019

Uh oh!

alexcrichton commented Aug 20, 2019

Uh oh!

bors commented Aug 20, 2019

Uh oh!

rust-highfive commented Aug 20, 2019

Uh oh!

jsgf commented Aug 22, 2019

Uh oh!

bors commented Aug 22, 2019

Uh oh!

bors commented Aug 22, 2019

Uh oh!

bors commented Aug 22, 2019

Uh oh!

jsgf commented Aug 22, 2019

Uh oh!

bors commented Aug 22, 2019

Uh oh!

bors commented Aug 22, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jsgf commented Jul 31, 2019•
edited
Loading

jsgf commented Jul 31, 2019•
edited
Loading

jsgf commented Aug 20, 2019•
edited
Loading