Still more `ObligationForest` improvements. by nnethercote · Pull Request #64805 · rust-lang/rust

nnethercote · 2019-09-26T07:19:31Z

Following on from #64627, more readability improvements, but negligible effects on speed.

nnethercote · 2019-09-26T07:22:23Z

This is the end of my spate of ObligationForest changes. I didn't manage to get rid of the memcpys caused by the swap calls within compress, which was the original motivation for looking into ObligationForest. (Well... not without (a) regressing performance, (b) causing test failures, or (c) using unsafe.) drain_filter() is the right fit conceptually, but caused a slowdown in practice :(

rust-highfive · 2019-09-26T09:05:09Z

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-09-26T07:20:08.1718424Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-26T07:20:08.1904525Z ##[command]git config gc.auto 0
2019-09-26T07:20:08.1998033Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-26T07:20:08.2060658Z ##[command]git config --get-all http.proxy
2019-09-26T07:20:08.2223453Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64805/merge:refs/remotes/pull/64805/merge
---
2019-09-26T08:27:04.2521920Z .................................................................................................... 1500/9044
2019-09-26T08:27:10.5786708Z .................................................................................................... 1600/9044
2019-09-26T08:27:23.4963227Z ..........................................................................i...............i......... 1700/9044
2019-09-26T08:27:30.6911019Z .................................................................................................... 1800/9044
2019-09-26T08:27:39.7858775Z .................................................................iiiii.............................. 1900/9044
2019-09-26T08:28:00.7517794Z .................................................................................................... 2100/9044
2019-09-26T08:28:03.4777073Z .................................................................................................... 2200/9044
2019-09-26T08:28:06.8425148Z .................................................................................................... 2300/9044
2019-09-26T08:28:15.9527139Z .................................................................................................... 2400/9044
---
2019-09-26T08:31:25.4262808Z ........................................................i...............i........................... 4700/9044
2019-09-26T08:31:35.2498035Z .................................................................................................... 4800/9044
2019-09-26T08:31:44.3020027Z .................................................................................................... 4900/9044
2019-09-26T08:31:52.2854541Z .................................................................................................... 5000/9044
2019-09-26T08:32:02.6221164Z ...........................................ii.ii.................................................... 5100/9044
2019-09-26T08:32:13.4139297Z .................................................................................................... 5300/9044
2019-09-26T08:32:24.4760863Z .................................................................................................... 5400/9044
2019-09-26T08:32:32.2826532Z ........i........................................................................................... 5500/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:50.6272060Z .................................................................................................... 5700/9044
2019-09-26T08:33:04.2621025Z ...ii...i..ii............i.......................................................................... 5800/9044
2019-09-26T08:33:27.3196761Z .................................................................................................... 6000/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:53.8213664Z .....i..ii.......................................................................................... 6200/9044
2019-09-26T08:34:14.1316332Z .................................................................i.................................. 6400/9044
2019-09-26T08:34:16.4682969Z .................................................................................................... 6500/9044
2019-09-26T08:34:19.2523721Z .....................................i.............................................................. 6600/9044
2019-09-26T08:34:23.5707037Z .................................................................................................... 6700/9044
---
2019-09-26T08:35:21.3521453Z .................................................................................................... 7200/9044
2019-09-26T08:35:27.1248564Z .................................................................................................... 7300/9044
2019-09-26T08:35:35.0538076Z .................................................................................................... 7400/9044
2019-09-26T08:35:46.0213214Z .................................................................................................... 7500/9044
2019-09-26T08:35:57.0818971Z .........................................................................................ii......i.. 7600/9044
2019-09-26T08:36:09.4301033Z .................................................................................................... 7800/9044
2019-09-26T08:36:27.7952234Z .................................................................................................... 7900/9044
2019-09-26T08:36:36.7016606Z .................................................................................................... 8000/9044
2019-09-26T08:36:47.4054420Z .................................................................................................... 8100/9044
---
2019-09-26T08:39:12.4658297Z finished in 5.766
2019-09-26T08:39:12.4806446Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:12.6761976Z 2019-09-26T08:39:12.6762234Z running 150 tests
2019-09-26T08:39:16.2542326Z i....iii......iii..iiii....i.............................i..i..................i....i.........ii.i.i 100/150
2019-09-26T08:39:18.3635693Z ..iiii..............i.........iii.i.......ii......
2019-09-26T08:39:18.3638395Z 2019-09-26T08:39:18.3643131Z finished in 5.883
2019-09-26T08:39:18.3829952Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:18.5572358Z ---
2019-09-26T08:39:20.7414149Z finished in 2.358
2019-09-26T08:39:20.7628910Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:20.9311079Z 2019-09-26T08:39:20.9338195Z running 9 tests
2019-09-26T08:39:20.9339024Z iiiiiiiii
2019-09-26T08:39:20.9339402Z 2019-09-26T08:39:20.9339482Z finished in 0.168
2019-09-26T08:39:20.9533216Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:21.1459137Z ---
2019-09-26T08:39:41.1289030Z finished in 20.174
2019-09-26T08:39:41.1575421Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:41.3785766Z 2019-09-26T08:39:41.3788665Z running 123 tests
2019-09-26T08:40:07.8578268Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-09-26T08:40:12.9879767Z i.i.i......iii.i.....ii
2019-09-26T08:40:12.9881676Z 2019-09-26T08:40:12.9887896Z finished in 31.834
2019-09-26T08:40:12.9901901Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:40:12.9902353Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-09-26T08:54:17.0429340Z 2019-09-26T08:54:17.0430114Z Doc-tests core
2019-09-26T08:54:22.5590872Z 2019-09-26T08:54:22.5591270Z running 2405 tests
2019-09-26T08:54:34.6884481Z ......iiiii......................................................................................... 100/2405
2019-09-26T08:54:46.6125539Z ................................................................................ii.................. 200/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:26.1961600Z ................................................i..i.................iiii........................... 500/2405
2019-09-26T08:55:48.6866484Z .................................................................................................... 700/2405
2019-09-26T08:56:00.2353757Z .................................................................................................... 800/2405
2019-09-26T08:56:11.5510730Z .................................................................................................... 900/2405
2019-09-26T08:56:22.8585477Z .................................................................................................... 1000/2405
---
2019-09-26T09:00:48.0831131Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:689:13
2019-09-26T09:00:48.0844677Z .......thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:00:48.0845128Z left: `1`,
2019-09-26T09:00:48.0846002Z right: `2`', src/libstd/sync/mutex.rs:653:13
2019-09-26T09:00:48.0913036Z ......thread '<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:791:13
2019-09-26T09:00:48.0930478Z ...thread '.<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:768:13
2019-09-26T09:00:48.0952645Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:705:13
2019-09-26T09:00:48.0961517Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:635:13
2019-09-26T09:00:48.0965441Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:646:13
2019-09-26T09:00:48.0972700Z ..thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:611:13
2019-09-26T09:00:50.1621647Z ..........................thread '<unnamed>' panicked at 'What the answer to my lifetimes dilemma is?', src/libstd/sys_common/remutex.rs:233:13
2019-09-26T09:00:50.1667530Z ........................ 700/763
2019-09-26T09:00:50.1772633Z ...................................thread '<unnamed>' panicked at 'explicit panic', src/libstd/thread/mod.rs:1535:13
2019-09-26T09:00:50.7859922Z ............thread '<unnamed>' panicked at 'Box<Any>', src/libstd/thread/mod.rs:1667:13
---
2019-09-26T09:00:57.5786006Z 2019-09-26T09:00:57.5787324Z running 992 tests
2019-09-26T09:01:20.7614874Z i................................................................................................... 100/992
2019-09-26T09:01:33.5081082Z .................................................................................................... 200/992
2019-09-26T09:01:42.4907211Z .................iii......i......i...i......i....................................................... 300/992
2019-09-26T09:01:48.7715508Z .................................................................................................... 400/992
2019-09-26T09:01:57.3470795Z ...................................i..i.................................ii.......................... 500/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:23.9195530Z ..................iiii.............................................................................. 800/992
2019-09-26T09:02:40.7113266Z .................................................................................................... 900/992
2019-09-26T09:02:49.2074246Z ........................................iiii................................................
2019-09-26T09:02:49.2075723Z 2019-09-26T09:02:49.2169224Z finished in 212.065
2019-09-26T09:02:49.2186343Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T09:02:49.4418675Z Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-09-26T09:05:03.4344108Z Finished release [optimized] target(s) in 17.89s
2019-09-26T09:05:03.4385425Z Running build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_data_structures-5f9a32e9b4b8b904
2019-09-26T09:05:03.4409830Z 2019-09-26T09:05:03.4410192Z running 154 tests
2019-09-26T09:05:03.4776721Z .................................FFFF.F............................................................. 100/154
2019-09-26T09:05:03.4905546Z failures:
2019-09-26T09:05:03.4905698Z 2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4908483Z thread 'obligation_forest::tests::diamond' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4908859Z left: `["A", "A.1", "A.2", "D"]`,
2019-09-26T09:05:03.4909384Z right: `["D", "A.2", "A.1", "A"]`', src/librustc_data_structures/obligation_forest/tests.rs:264:5
2019-09-26T09:05:03.4909770Z 2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910634Z thread 'obligation_forest::tests::done_dependency' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4911321Z left: `["A: Sized", "B: Sized", "C: Sized"]`,
2019-09-26T09:05:03.4911906Z right: `["C: Sized", "B: Sized", "A: Sized"]`', src/librustc_data_structures/obligation_forest/tests.rs:326:5
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912941Z thread 'obligation_forest::tests::orphan' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4913133Z left: `["C1", "C2"]`,
2019-09-26T09:05:03.4913530Z right: `["C2", "C1"]`', src/librustc_data_structures/obligation_forest/tests.rs:364:5
2019-09-26T09:05:03.4913725Z 2019-09-26T09:05:03.4914108Z ---- obligation_forest::tests::push_pop stdout ----
2019-09-26T09:05:03.4914544Z thread 'obligation_forest::tests::push_pop' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4914753Z left: `["A.1", "A.3", "A.3.i"]`,
2019-09-26T09:05:03.4915187Z right: `["A.3", "A.1", "A.3.i"]`', src/librustc_data_structures/obligation_forest/tests.rs:119:5
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4916221Z thread 'obligation_forest::tests::success_in_grandchildren' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4916432Z left: `["A.1", "A.3"]`,
2019-09-26T09:05:03.4916833Z right: `["A.3", "A.1"]`', src/librustc_data_structures/obligation_forest/tests.rs:175:5
2019-09-26T09:05:03.4917230Z 2019-09-26T09:05:03.4917505Z failures:
2019-09-26T09:05:03.4917640Z obligation_forest::tests::diamond
2019-09-26T09:05:03.4917789Z obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4917789Z obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4918304Z obligation_forest::tests::orphan
2019-09-26T09:05:03.4918447Z obligation_forest::tests::push_pop
2019-09-26T09:05:03.4918606Z obligation_forest::tests::success_in_grandchildren
2019-09-26T09:05:03.4918728Z 2019-09-26T09:05:03.4918870Z test result: FAILED. 149 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out
2019-09-26T09:05:03.4918992Z 2019-09-26T09:05:03.4919454Z error: test failed, to rerun pass '-p rustc_data_structures --lib'
2019-09-26T09:05:03.4963972Z 2019-09-26T09:05:03.4965023Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_data_structures" "--" "--quiet"
2019-09-26T09:05:03.4966835Z expected success, got: exit code: 101
2019-09-26T09:05:03.4967760Z 2019-09-26T09:05:03.4967760Z 2019-09-26T09:05:03.4967936Z 2019-09-26T09:05:03.4968094Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-09-26T09:05:03.4971688Z Build completed unsuccessfully in 1:37:14
2019-09-26T09:05:03.4972006Z == clock drift check ==
2019-09-26T09:05:03.4998863Z local time: Thu Sep 26 09:05:03 UTC 2019
2019-09-26T09:05:03.6527725Z network time: Thu, 26 Sep 2019 09:05:03 GMT
2019-09-26T09:05:03.6530378Z == end clock drift check ==
2019-09-26T09:05:04.3452982Z ##[error]Bash exited with code '1'.
2019-09-26T09:05:04.3503490Z ##[section]Starting: Checkout
2019-09-26T09:05:04.3505447Z ==============================================================================
2019-09-26T09:05:04.3505518Z Task : Get sources
2019-09-26T09:05:04.3505566Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

pnkfelix · 2019-09-26T09:06:01Z

@nnethercote what sort of test failures were you observing from (b) above?

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

nnethercote · 2019-09-26T11:08:21Z

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

They were all changes in error messages that arose from changing the order of pending obligations. Some of them looked pretty trivial (e.g. change the order of two errors) but some of them were more complex, to the point where I was struggling to tell if the new messages were as valid as the old ones. And there were quite a few in the latter basket. So I ended up preserving the order because the potential speed benefits of reordering were small.

nnethercote · 2019-09-26T11:09:35Z

The test failures on automation are trivial; some vectors of erroneous obligations have changed order, but (AIUI) the order is not significant. I will fix those on Monday.

It's not necessary; cycles (which are rare) can be detected by looking at the node stack. This change speeds things up slightly, as well as simplifying the code a little.

For consistency with other variables in this file.

It has a single call site, and the code is easier to read this way.

They don't help performance at all, and just complicate things.

These make the code more concise.

By collecting the done obligations (when necessary) in the main loop. This makes the code cleaner. The commit also changes the order in which successful obligations are returned -- they are now returned in the registered order, rather than reversed. Because this order doesn't actually matter, being only used by tests, the commit uses `sort()` to make the test agnostic w.r.t. the order.

nnethercote · 2019-09-29T20:37:16Z

I fixed the test failures and rebased. Ready for review, @nikomatsakis!

nikomatsakis · 2019-10-01T20:39:09Z

@bors r+

bors · 2019-10-01T20:39:10Z

📌 Commit a820672 has been approved by nikomatsakis

@nikomatsakis

… r=nikomatsakis Still more `ObligationForest` improvements. Following on from rust-lang#64627, more readability improvements, but negligible effects on speed. r? @nikomatsakis

@ghost

Rollup of 11 pull requests Successful merges: - #64649 (Avoid ICE on return outside of fn with literal array) - #64722 (Make all alt builders produce parallel-enabled compilers) - #64801 (Avoid `chain()` in `find_constraint_paths_between_regions()`.) - #64805 (Still more `ObligationForest` improvements.) - #64840 (SelfProfiler API refactoring and part one of event review) - #64885 (use try_fold instead of try_for_each to reduce compile time) - #64942 (Fix clippy warnings) - #64952 (Update cargo.) - #64974 (Fix zebra-striping in generic dataflow visualization) - #64978 (Fully clear `HandlerInner` in `Handler::reset_err_count`) - #64979 (Update books) Failed merges: - #64959 (syntax: improve parameter without type suggestions) r? @ghost

rust-highfive assigned nikomatsakis Sep 26, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 26, 2019

nnethercote added 11 commits September 30, 2019 05:23

Remove NodeState::OnDfsStack.
3b506d9
It's not necessary; cycles (which are rare) can be detected by looking at the node stack. This change speeds things up slightly, as well as simplifying the code a little.

Rename node_index as index.
4a7fb8b
For consistency with other variables in this file.

Inline mark_as_waiting_from.
85a56cb
It has a single call site, and the code is easier to read this way.

Tweak the control flow in process_obligations().
7130e6c

Use filter and map in to_errors.
ea72650

Remove unnecessary uses of ObligationForest::scratch.
22943ee
They don't help performance at all, and just complicate things.

Introduce some intermediate variables that aid readability.
6fb1f37

Convert some match expressions to ifs.
9e67f19
These make the code more concise.

Rename nodes_len and use it in a few more places.
8a62bb1

Remove an out-of-date sentence in a comment.
2883c25

nnethercote force-pushed the ObligForest-still-more branch from 3af2b25 to a820672Compare September 29, 2019 20:30

nikomatsakis approved these changes Oct 1, 2019
View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 1, 2019

tmandry mentioned this pull request Oct 2, 2019
Rollup of 11 pull requests #64981
Merged

bors merged commit a820672 into rust-lang:masterOct 2, 2019

nnethercote deleted the ObligForest-still-more branch April 2, 2020 03:48

Navigation Menu

Uh oh!

Uh oh!

Still more `ObligationForest` improvements. - #64805

Still more `ObligationForest` improvements.#64805
bors merged 11 commits into
rust-lang:masterfrom
nnethercote:ObligForest-still-more

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

rust-highfive commented Sep 26, 2019

Uh oh!

pnkfelix commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 29, 2019

Uh oh!

nikomatsakis commented Oct 1, 2019

Uh oh!

bors commented Oct 1, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Uh oh!

Conversation

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

rust-highfive commented Sep 26, 2019

Uh oh!

pnkfelix commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 26, 2019

Uh oh!

nnethercote commented Sep 29, 2019

Uh oh!

nikomatsakis commented Oct 1, 2019

Uh oh!

bors commented Oct 1, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants