add target features when extracting and running doctests by QuietMisdreavus · Pull Request #49864 · rust-lang/rust

QuietMisdreavus · 2018-04-11T00:27:00Z

When rendering documentation, rustdoc will happily load target features into the cfg environment from the current target, but fails to do this when doing anything with doctests. This would lead to situations where, thanks to #48759, functions tagged with #[target_feature] couldn't run doctests, thanks to the automatic #[doc(cfg(target_feature = "..."))].

Currently, there's no way to pass codegen options to rustdoc that will affect its rustc sessions, but for now this will let you use target features that come default on the platform you're targeting.

Fixes#49723

rust-highfive · 2018-04-11T00:27:03Z

r? @steveklabnik

(rust_highfive has picked a reviewer for you, use r? to override)

QuietMisdreavus · 2018-04-11T00:27:21Z

r? @GuillaumeGomez

TimNN · 2018-04-11T03:06:07Z

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:00:47] configure: rust.quiet-tests := True
---
[00:44:47] ..............................................................................i.....................
[00:44:53] .....................i..............................................................................
---
[00:45:36] i..........................................................................i........................
---
[00:46:33] .............................................i......................................................
---
[00:50:33] .............................i......................................................................
[00:50:48] ..............................................................i.....................................
[00:51:03] ...............................................i....................................................
[00:51:24] ....................................................................................................
[00:51:46] ....................................................................................................
[00:52:08] ....................................................................................................
[00:52:34] ...i...............................................................................................i
[00:53:01] .............................................................................test [run-pass] run-pass/mir_heavy_promoted.rs has been running for over 60 seconds
[00:53:12] .......................
[00:53:43] ....................................................................................................
[00:54:19] .................................................................ii.................................
[00:55:05] ............................i....................................................i.ii.test [run-pass] run-pass/saturating-float-casts.rs has been running for over 60 seconds
[00:55:12] ..............
[00:55:55] .........................................................................................iiiiiii....
---
[00:58:10] ...................i............................................................ii.iii..............
[00:58:17] ....................................................................................................
[00:58:25] .........i..............................i...........................................................
[00:58:33] ....................................................................................................
[00:58:40] .....................i..............................................................................
[00:58:48] ....................................................................................................
[00:58:58] ....................................................................................................
[00:59:08] ....................................................................................................
[00:59:19] ....................................................................................................
[00:59:33] ....................................................................................................
[00:59:42] ..............i.....................................................................................
[00:59:52] ..................i..ii.............................................................................
[01:00:02] ....................................................................................................
[01:00:12] ....................................................................................................
[01:00:22] .....................................................................................i..............
[01:00:32] ...............................i....................................................................
---
[01:01:10] ...........................i........................................................................
[01:01:12] ....................................................................i...............................
[01:01:13] ................i.......................................................
---
[01:01:28] .............i........................
---
[01:01:59] i...i..ii....i.............ii.........iii......i..i...i...ii..i..i..ii.....
---
[01:02:02] i.......i......................i.......
---
[01:02:41] iiii.......i..i........i..i.i.............i..........iiii...........i...i..........ii.i.i.......ii..
[01:02:42] ....ii...
---
[01:10:31] ....................F...............................................................................
[01:12:07] .......i............................................................................................
---
[01:12:30] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/rustdoc" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-3.9/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zmiri -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zmiri -Zunstable-options -Lnative=/checnknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu
---
56592 ./obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/incremental/syntax-33oa6nnkk1g08/s-f006tf4qli-10t4q2j-v8bl3ay9sjlv

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

GuillaumeGomez · 2018-04-11T08:20:12Z

It'd be actually nice to pass compile options to tests... We should definitely talk about it!

gnzlbg · 2018-04-11T08:34:53Z

Thanks, I think that for the time being this is a good solution!

It allows users to at least provide their own hidden environment (main, #[target_feature], use static/run-time feature detection, etc.) and that is something that we will always want to provide for maximum control.

It'd be actually nice to pass compile options to tests... We should definitely talk about it!

If cargo test would set the RUSTFLAGS environment variable for doc tests that would already be great.

QuietMisdreavus · 2018-04-11T13:58:13Z

Just passing RUSTFLAGS to doctests won't work - rustdoc handles the compilation of doctests itself, which bypasses any other command-line flags that you may want to hand to rustc. Rustdoc would need to handle the -C flag itself and pass those codegen options on when setting up the tests.

QuietMisdreavus · 2018-04-11T20:22:32Z

I was able to get this test to work locally, so i'm blaming the CI environment. Is there something i'm not seeing here? @gnzlbg I thought the sse feature was supposed to be always enabled on x86_64?

gnzlbg · 2018-04-12T17:53:09Z

If this doctest fails, is because foo was not part of the binary, so... "sse" is not set for some reason, which is very weird. It is set for x86_64 always. cc @alexcrichton

Hm I'm not sure why this isn't working, rustc +nightly --print cfg clearly shows it's there.

If you run the test locally (i.e. x.py test --stage 1 src/test/rustdoc --test-args "target-feature") then it will work properly, implying there's something off with the Travis setup.

Aha right yes, you can tag this as // no-system-llvm

QuietMisdreavus · 2018-04-12T20:59:03Z

Aha right yes, you can tag this as // no-system-llvm

Well, this makes travis skip the test at least. It still runs and passes locally on my system, too, so hopefully it works this time.

I've reverted the test to the original version (that has feature detection both on the item and in the test) and squashed it down. If travis still agrees (i.e. skips the test) then this PR is ready. I have a local branch waiting for this to add a -C flag to rustdoc to allow for setting codegen options, including manually setting features.

QuietMisdreavus · 2018-04-12T22:26:45Z

Alright, passed travis! @GuillaumeGomez ready for review!

gnzlbg · 2018-04-13T10:29:53Z

@alexcrichton what was the problem? IIRC there is some stuff with target features that we are only doing with Rust's LLVM, was that it?

alexcrichton · 2018-04-13T14:05:11Z

We can only read a list of target features from our own LLVM, not system LLVMs

GuillaumeGomez · 2018-04-14T10:30:49Z

Let's go then!

@bors: r+

bors · 2018-04-14T10:30:50Z

📌 Commit 3366032 has been approved by GuillaumeGomez

…ures, r=GuillaumeGomez add target features when extracting and running doctests When rendering documentation, rustdoc will happily load target features into the cfg environment from the current target, but fails to do this when doing anything with doctests. This would lead to situations where, thanks to rust-lang#48759, functions tagged with `#[target_feature]` couldn't run doctests, thanks to the automatic `#[doc(cfg(target_feature = "..."))]`. Currently, there's no way to pass codegen options to rustdoc that will affect its rustc sessions, but for now this will let you use target features that come default on the platform you're targeting. Fixesrust-lang#49723

Rollup of 14 pull requests Successful merges: #49908, #49876, #49916, #49951, #49465, #49922, #49866, #49915, #49886, #49913, #49852, #49958, #49871, #49864 Failed merges:

rustdoc: port the -C option from rustc Blocked on #49864. The included test won't work without those changes, so this PR includes those commits as well. When documenting items that require certain target features, it helps to be able to force those target features into existence. Rather than include a flag just to parse those features, i instead decided to port the `-C` flag from rustc in its entirety. It takes the same parameters, because it runs through the same parsing function. This has the added benefit of being able to control the codegen of doctests as well. One concern i have with the flag is that i set it to stable here. My rationale is that it is a direct port of functionality on rustc that is currently stable, used only in mechanisms that it is originally used for. If needed, i can set it back to be unstable.

add target features when extracting and running doctests
43301e5

rust-highfive assigned steveklabnik Apr 11, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 11, 2018

rust-highfive assigned GuillaumeGomez and unassigned steveklabnik Apr 11, 2018

QuietMisdreavus mentioned this pull request Apr 11, 2018
rustdoc doesn't execute documentation tests for #[target_feature] functions #49723
Closed

QuietMisdreavus force-pushed the doctest-target-features branch from e7ea77d to 3b1c3d4Compare April 12, 2018 15:16

gnzlbg reviewed Apr 12, 2018
View reviewed changes

QuietMisdreavus force-pushed the doctest-target-features branch from 3b1c3d4 to 9e0e263Compare April 12, 2018 18:39

add test for using target features in doctests
3366032

QuietMisdreavus force-pushed the doctest-target-features branch from 9e0e263 to 3366032Compare April 12, 2018 20:51

QuietMisdreavus mentioned this pull request Apr 13, 2018
rustdoc: port the -C option from rustc #49956
Merged

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 14, 2018

kennytm mentioned this pull request Apr 14, 2018
Rollup of 14 pull requests #49939
Merged

bors added a commit that referenced this pull request Apr 14, 2018
Auto merge of#49939- kennytm:rollup, r=kennytm
21dae95
Rollup of 14 pull requests Successful merges: #49908, #49876, #49916, #49951, #49465, #49922, #49866, #49915, #49886, #49913, #49852, #49958, #49871, #49864 Failed merges:

bors merged commit 3366032 into rust-lang:masterApr 14, 2018

QuietMisdreavus deleted the doctest-target-features branch May 9, 2018 20:57

Uh oh!

Uh oh!

Conversation

QuietMisdreavus commented Apr 11, 2018

Uh oh!

rust-highfive commented Apr 11, 2018

Uh oh!

QuietMisdreavus commented Apr 11, 2018

Uh oh!

TimNN commented Apr 11, 2018

Uh oh!

GuillaumeGomez commented Apr 11, 2018

Uh oh!

gnzlbg commented Apr 11, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuietMisdreavus commented Apr 11, 2018

Uh oh!

QuietMisdreavus commented Apr 11, 2018

Uh oh!

gnzlbgApr 12, 2018

Choose a reason for hiding this comment

Uh oh!

alexcrichtonApr 12, 2018

Choose a reason for hiding this comment

Uh oh!

QuietMisdreavusApr 12, 2018

Choose a reason for hiding this comment

Uh oh!

alexcrichtonApr 12, 2018

Choose a reason for hiding this comment

Uh oh!

QuietMisdreavus commented Apr 12, 2018

Uh oh!

QuietMisdreavus commented Apr 12, 2018

Uh oh!

gnzlbg commented Apr 13, 2018

Uh oh!

alexcrichton commented Apr 13, 2018

Uh oh!

GuillaumeGomez commented Apr 14, 2018

Uh oh!

bors commented Apr 14, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

gnzlbg commented Apr 11, 2018•
edited
Loading