Anonymize some line numbers in UI test output by petrochenkov · Pull Request #48449 · rust-lang/rust

petrochenkov · 2018-02-23T00:51:02Z

New unstable flag -Z ui-testing is introduced. This flag changes diagnostic output of the compiler in some way making it more suitable for UI testing (this is intentionally vague).
At the moment this flag anonymizes line numbers at line starts thus solving the largest issue with UI test diffs. If diffs continue to be too noisy, some other tweaks could be applied (e.g. anonymizing lines/columns in --> $DIR/file.rs:line:column), but this needs some time and experience (we shouldn't diverge too much from the actual output in general).

If comment // disable-ui-testing-normalization is added to an UI test, then -Z ui-testing is not passed.

Closes#46643

rust-highfive · 2018-02-23T00:51:06Z

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2018-02-23T00:53:13Z

r? @nikomatsakis

petrochenkov · 2018-02-23T01:00:00Z

Example of a test with "long" actual numbers that become "short" anonymized numbers.

estebank · 2018-02-23T02:07:07Z

This doesn't deal with the line number in the file:line:col section, but should be fine. LGTM.

oli-obk · 2018-02-23T09:07:45Z

There's a line number left here

Yeah, noticed these too late, will fix today or tomorrow.
I haven't figured out yet where they come from.

petrochenkov · 2018-02-23T10:05:13Z

This doesn't deal with the line number in the file:line:col section

cc #46643 (comment)

bors · 2018-02-23T14:27:06Z

☔ The latest upstream changes (presumably #47799) made this pull request unmergeable. Please resolve the merge conflicts.

petrochenkov · 2018-02-23T22:00:43Z

Updated.

nikomatsakis · 2018-02-23T22:07:39Z

Now that we have the //~ ERROR lines, sgtm.

@bors r+

bors · 2018-02-23T22:07:39Z

📌 Commit a1065b4 has been approved by nikomatsakis

bors · 2018-02-23T22:07:40Z

🌲 The tree is currently closed for pull requests below priority 99, this pull request will be tested once the tree is reopened

petrochenkov · 2018-02-23T22:12:56Z

@bors p=1
The patch is conflict-prone + other PRs in the queue modifying UI tests will benefit from it.

bors · 2018-02-24T10:40:15Z

☔ The latest upstream changes (presumably #48476) made this pull request unmergeable. Please resolve the merge conflicts.

petrochenkov · 2018-02-24T21:04:20Z

@bors r=nikomatsakis

bors · 2018-02-24T21:04:20Z

📌 Commit b94e8a4 has been approved by nikomatsakis

bors · 2018-02-24T21:04:21Z

🌲 The tree is currently closed for pull requests below priority 100, this pull request will be tested once the tree is reopened

Mark-Simulacrum · 2018-02-26T22:10:57Z

Looks like now 64-bit AppVeyor is starting to hit the timeout as well :/

kennytm · 2018-02-26T23:55:05Z

check-aux is timing out quite a lot actually. Perhaps doing cargo-test + pretty test together is too long for Windows. I’m thinking of moving one of the check-aux tests to the tools builder, but we need to figure out which ones can balance the time.

bors · 2018-02-27T00:29:57Z

⌛ Testing commit 9f9183d with merge 29f5c69...

Anonymize some line numbers in UI test output New unstable flag `-Z ui-testing` is introduced. This flag changes diagnostic output of the compiler *in some way* making it more suitable for UI testing (this is intentionally vague). At the moment this flag anonymizes line numbers at line starts thus solving the largest issue with UI test diffs. If diffs continue to be too noisy, some other tweaks could be applied (e.g. anonymizing lines/columns in `--> $DIR/file.rs:line:column`), but this needs some time and experience (we shouldn't diverge too much from the actual output in general). If comment `// disable-ui-testing-normalization` is added to an UI test, then `-Z ui-testing` is not passed. Closes#46643

bors · 2018-02-27T03:16:01Z

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing 29f5c69 to master...

This reverts commit 1dc2015, which switched to compiling the UI tests twice rather than extracting the humanized output from a field in the JSON output. A conflict in this revert commit had to be fixed manually where changes introduced in rust-lang#48449 collide with the change we're trying to revert (from rust-lang#48337). This is in the matter of rust-lang#48550. Conflicts: src/tools/compiletest/src/runtest.rs

This reverts commit 9b597a1. That commit added a message informing users about the `rustc --explain` functionality inside of a `Drop` implementation for `EmitterWriter`. In addition to exhibiting questionable semantics (printing a hopefully-helpful message for the user does not seem like a "cleanup" action), this resulted in divergent behavior between humanized output and JSON output, because the latter actually instantiates an `EmitterWriter` for every diagnostic. Several conflicts in this revert commit had to be fixed manually, again owing to code-area collision between rust-lang#48449 and rust-lang#48337. This is in the matter of rust-lang#48550. Conflicts: src/librustc_errors/emitter.rs

rust-highfive assigned estebank Feb 23, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 23, 2018

rust-highfive assigned nikomatsakis and unassigned estebank Feb 23, 2018

petrochenkov commented Feb 23, 2018
View reviewed changes

oli-obk reviewed Feb 23, 2018
View reviewed changes

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 23, 2018

petrochenkov force-pushed the uidiff branch from 44cbe5d to a1065b4Compare February 23, 2018 21:57

petrochenkov added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 23, 2018

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 23, 2018

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 24, 2018

petrochenkov force-pushed the uidiff branch from a1065b4 to b94e8a4Compare February 24, 2018 21:03

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Feb 26, 2018

bors mentioned this pull request Feb 27, 2018
Reword E0044 and message for !Send types #48138
Merged

bors merged commit 9f9183d into rust-lang:masterFeb 27, 2018

estebank mentioned this pull request Feb 28, 2018
Suggest type for overflowing bin/hex-literals #48432
Merged

zackmdavis mentioned this pull request Feb 28, 2018
in which parentheses are suggested for should-have-been-tuple-patterns #48527
Merged

kennytm mentioned this pull request Feb 28, 2018
[Underspecified semantics] Type check defaults at declaration. #46785
Merged

emilyalbini mentioned this pull request Feb 28, 2018
Use free regions when determining self type in compare_impl_method#48419
Merged

flip1995 added a commit to flip1995/rust that referenced this pull request Mar 1, 2018
Adapt stderr of UI test to PRrust-lang#48449
f45f760

petrochenkov deleted the uidiff branch June 5, 2019 16:02

Uh oh!

Uh oh!

Conversation

petrochenkov commented Feb 23, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented Feb 23, 2018

Uh oh!

petrochenkov commented Feb 23, 2018

Uh oh!

petrochenkovFeb 23, 2018

Choose a reason for hiding this comment

Uh oh!

estebank commented Feb 23, 2018

Uh oh!

oli-obkFeb 23, 2018

Choose a reason for hiding this comment

Uh oh!

petrochenkovFeb 23, 2018

Choose a reason for hiding this comment

Uh oh!

petrochenkov commented Feb 23, 2018

Uh oh!

bors commented Feb 23, 2018

Uh oh!

petrochenkov commented Feb 23, 2018

Uh oh!

nikomatsakis commented Feb 23, 2018

Uh oh!

bors commented Feb 23, 2018

Uh oh!

bors commented Feb 23, 2018

Uh oh!

petrochenkov commented Feb 23, 2018

Uh oh!

bors commented Feb 24, 2018

Uh oh!

petrochenkov commented Feb 24, 2018

Uh oh!

bors commented Feb 24, 2018

Uh oh!

bors commented Feb 24, 2018

Uh oh!

Mark-Simulacrum commented Feb 26, 2018

Uh oh!

kennytm commented Feb 26, 2018

Uh oh!

bors commented Feb 27, 2018

Uh oh!

bors commented Feb 27, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

petrochenkov commented Feb 23, 2018•
edited
Loading