Optimize HIR map by Zoxc · Pull Request #60246 · rust-lang/rust

Zoxc · 2019-04-24T19:47:28Z

Builds on #59042

Zoxc · 2019-04-24T20:01:55Z

bors · 2019-04-24T20:02:07Z

⌛ Trying commit d3bd03ce1d50238d54a70a96b22731cd76cc5e20 with merge 890cc87d1001f65c608c711d7a4d0d48129a43c1...

rust-highfive · 2019-04-24T21:46:22Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:001763f4:start=1556135562249149308,finish=1556135563013292812,duration=764143504
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
travis_time:start:test_assembly
Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:22:24] [01:22:24] running 9 tests
[01:22:24] iiiiiiiii
[01:22:24] [01:22:24] finished in 0.151
[01:22:24] travis_fold:end:test_assembly
---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:22:40] [01:22:40] running 121 tests
[01:23:06] .iiiii...i.....i..i...i..i.i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i. 100/121
[01:23:10] i.i......iii.i.....ii
[01:23:10] [01:23:10] finished in 30.681
[01:23:10] travis_fold:end:test_debuginfo
---
Check compiletest suite=run-make-fulldeps mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:52:08] [01:52:08] running 197 tests
[01:52:37] ..................i...i................................................................i............ 100/197
[01:53:23] ...................................F.................i......................................i....
[01:53:23] failures:
[01:53:23] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:53:23] [01:53:23] ---- [run-make] run-make-fulldeps/pretty-print-path-suffix stdout ----
[01:53:23] ---- [run-make] run-make-fulldeps/pretty-print-path-suffix stdout ----
[01:53:23] [01:53:23] error: make failed
[01:53:23] status: exit code: 2
[01:53:23] command: "make"
[01:53:23] stdout:
[01:53:23] ------------------------------------------
[01:53:23] make[1]: Entering directory '/checkout/src/test/run-make-fulldeps/pretty-print-path-suffix'
[01:53:23] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix/foo.out -Z unpretty=hir=foo input.rs
[01:53:23] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix/nest_foo.out -Z unpretty=hir=nest::foo input.rs
[01:53:23] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix/foo_method.out -Z unpretty=hir=foo_method input.rs
[01:53:23] diff -u /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix/foo.out foo.pp
[01:53:23] --- /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/pretty-print-path-suffix/pretty-print-path-suffix/foo.out 2019-04-24 21:45:41.689211650 +0000
[01:53:23] +++ foo.pp 2019-04-24 19:52:41.805211650 +0000
[01:53:23] @@ -0,0 +1,5 @@
[01:53:23] +
[01:53:23] +pub fn foo() -> i32 { 45 } /* foo */
[01:53:23] +
[01:53:23] +
[01:53:23] +pub fn foo() -> &'static str { "i am a foo." } /* nest::foo */
[01:53:23] Makefile:4: recipe for target 'all' failed
[01:53:23] make[1]: Leaving directory '/checkout/src/test/run-make-fulldeps/pretty-print-path-suffix'
[01:53:23] ------------------------------------------
[01:53:23] stderr:
[01:53:23] ------------------------------------------
[01:53:23] ------------------------------------------
[01:53:23] warning: ignoring --out-dir flag due to -o flag
[01:53:23] [01:53:23] warning: ignoring --out-dir flag due to -o flag
[01:53:23] [01:53:23] warning: ignoring --out-dir flag due to -o flag
[01:53:23] [01:53:23] make[1]: *** [all] Error 1
[01:53:23] ------------------------------------------
[01:53:23] [01:53:23] [01:53:23] [01:53:23] [01:53:23] failures:
[01:53:23] [run-make] run-make-fulldeps/pretty-print-path-suffix
[01:53:23] [01:53:23] test result: FAILED. 191 passed; 1 failed; 5 ignored; 0 measured; 0 filtered out
[01:53:23] [01:53:23] [01:53:23] [01:53:23] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--llvm-components" "aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter binaryformat bitreader bitwriter bpf bpfasmparser bpfasmprinter bpfcodegen bpfdesc bpfdisassembler bpfinfo codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfomsf debuginfopdb demangle dlltooldriver engine executionengine fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader lanai lanaiasmparser lanaiasmprinter lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize windowsmanifest x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo" "--llvm-cxxflags" "-I/usr/lib/llvm-6.0/include -std=c++0x -fuse-ld=gold -Wl,--no-keep-files-mapped -Wl,--no-map-whole-files -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O2 -DNDEBUG -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS" "--ar" "ar" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:53:23] [01:53:23] [01:53:23] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:53:23] Build completed unsuccessfully in 0:42:50
[01:53:23] Build completed unsuccessfully in 0:42:50
[01:53:23] make: *** [check] Error 1
[01:53:23] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:1004690c
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed Apr 24 21:46:16 UTC 2019

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-04-24T22:31:44Z

☀️ Try build successful - checks-travis
Build commit: 890cc87d1001f65c608c711d7a4d0d48129a43c1

Xanewok · 2019-04-24T22:52:29Z

@rust-timer build 890cc87d1001f65c608c711d7a4d0d48129a43c1

rust-timer · 2019-04-24T22:52:30Z

Success: Queued 890cc87d1001f65c608c711d7a4d0d48129a43c1 with parent e305df1, comparison URL.

rust-timer · 2019-04-25T03:07:05Z

Finished benchmarking try commit 890cc87d1001f65c608c711d7a4d0d48129a43c1

Zoxc · 2019-04-25T03:59:34Z

Seems like this approach works well. There's still some slight memory regressions though.

rust-highfive · 2019-04-25T18:36:55Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:005982c6:start=1556216519294202627,finish=1556216521438908215,duration=2144705588
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[00:04:18] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:19] tidy error: /checkout/src/librustc/hir/map/mod.rs:1215: trailing whitespace
[00:04:20] some tidy checks failed
[00:04:20] [00:04:20] [00:04:20] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:20] [00:04:20] [00:04:20] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:20] Build completed unsuccessfully in 0:00:45
[00:04:20] Build completed unsuccessfully in 0:00:45
[00:04:20] make: *** [tidy] Error 1
[00:04:20] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0ec45cf8
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Thu Apr 25 18:26:33 UTC 2019
---
travis_time:end:0d80e979:start=1556216793919279285,finish=1556216793924162685,duration=4883400
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0d377e70
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:10837696
travis_time:start:10837696
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:1634df17
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Xanewok

IIUC this fixes a regression introduced in #59042 and doesn't look like a very invasive change (limited only to one data structure) - should we not land it?

Xanewok · 2019-05-01T09:40:43Z

It takes a while to decipher what the structure looks like and why it looks this way - it'd be great if we could add a doc-comment here with an explanation

Xanewok · 2019-05-01T09:41:49Z

It'd be great to also add a comment here and/or rustfmt the function body

bors · 2019-05-06T16:47:02Z

☔ The latest upstream changes (presumably #60337) made this pull request unmergeable. Please resolve the merge conflicts.

eddyb · 2019-05-06T17:46:36Z

hir::map::Mapis the HIR map. Can you rename this type alias to HirEntryMap or something?

eddyb

This looks great, r=me modulo nits!

Zoxc · 2019-05-08T12:29:52Z

@bors r=eddyb

bors · 2019-05-08T12:29:53Z

📌 Commit d33db6e has been approved by eddyb

bors · 2019-05-08T12:30:04Z

⌛ Testing commit d33db6e with merge d7f4a86...

@ljedrz

Optimize HIR map Builds on #59042 cc @ljedrz r? @eddyb

bors · 2019-05-08T15:24:09Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: eddyb
Pushing d7f4a86 to master...

bjorn3 · 2019-05-08T16:08:13Z

- return Some(idx)
- }
- }
+ fn matces_suffix(&self, hir: HirId) -> bool {


bjorn3 · 2019-05-08T16:08:35Z

- }
+ };
+
+ self.all_ids().filter(move |hir| nodes.matces_suffix(*hir)).map(move |hir| {


bjorn3 · 2019-05-08T16:12:24Z

I am a little confused about the order of comments on this PR: bors merges (1h ago) -> me reviewing it (few min ago) -> Zoxc gives r+ (3h ago).

Zoxc · 2019-05-08T16:41:35Z

@bjorn3 It's github new feature, time-travelling comments.

bjorn3 · 2019-05-08T16:52:04Z

When looking at my notifications list, it said bors was the last commenter :)

Zoxc · 2019-05-08T16:58:58Z

You can see that I will post my comment in 3 hours. Github simply sorts comments by time!

bjorn3 · 2019-05-08T17:27:46Z

On mobile it says 2 hours ago though.

Zoxc · 2019-05-14T18:33:20Z

Perf result: https://perf.rust-lang.org/compare.html?start=b92d360c6cf029bd98c154cb510ec9e11b39bad6&end=d7f4a864e13dc88d7a28c0b742da844af4e3abc4&stat=wall-time

rust-highfive assigned eddyb Apr 24, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 24, 2019

Zoxc changed the title ~~Optimize HIR map~~[WIP] Optimize HIR mapApr 25, 2019

Zoxc force-pushed the hir-map-vec branch from d3bd03c to fcfe0f6Compare April 25, 2019 18:20

Zoxc force-pushed the hir-map-vec branch from fcfe0f6 to f2ba7efCompare April 25, 2019 18:43

Zoxc changed the title ~~[WIP] Optimize HIR map~~Optimize HIR mapApr 25, 2019

Xanewok reviewed May 1, 2019
View reviewed changes

eddyb reviewed May 6, 2019
View reviewed changes

eddyb suggested changes May 6, 2019
View reviewed changes

Zoxc added 2 commits May 8, 2019 12:18

Optimize HIR map
df5a011

Rename HirMap to HirEntryMap and add some comments
d33db6e

Zoxc force-pushed the hir-map-vec branch from f2ba7ef to d33db6eCompare May 8, 2019 12:28

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 8, 2019

bors added a commit that referenced this pull request May 8, 2019
Auto merge of#60246- Zoxc:hir-map-vec, r=eddyb
d7f4a86
Optimize HIR map Builds on #59042 cc @ljedrz r? @eddyb

bors added the merged-by-bors This PR was explicitly merged by bors. label May 8, 2019

bors merged commit d33db6e into rust-lang:masterMay 8, 2019

bors mentioned this pull request May 8, 2019
Use Symbol more #60630
Merged

bjorn3 reviewed May 8, 2019
View reviewed changes

Zoxc deleted the hir-map-vec branch May 8, 2019 16:41

bors mentioned this pull request May 8, 2019
submodules: update clippy from fc96aa03 to 341c96af #60626
Merged

Uh oh!

Uh oh!

Conversation

Zoxc commented Apr 24, 2019

Uh oh!

Zoxc commented Apr 24, 2019

Uh oh!

bors commented Apr 24, 2019

Uh oh!

rust-highfive commented Apr 24, 2019

Uh oh!

bors commented Apr 24, 2019

Uh oh!

Xanewok commented Apr 24, 2019

Uh oh!

rust-timer commented Apr 24, 2019

Uh oh!

rust-timer commented Apr 25, 2019

Uh oh!

Zoxc commented Apr 25, 2019

Uh oh!

rust-highfive commented Apr 25, 2019

Uh oh!

Xanewok left a comment

Choose a reason for hiding this comment

Uh oh!

XanewokMay 1, 2019

Choose a reason for hiding this comment

Uh oh!

XanewokMay 1, 2019

Choose a reason for hiding this comment

Uh oh!

bors commented May 6, 2019

Uh oh!

eddybMay 6, 2019

Choose a reason for hiding this comment

Uh oh!

eddyb left a comment

Choose a reason for hiding this comment

Uh oh!

Zoxc commented May 8, 2019

Uh oh!

bors commented May 8, 2019

Uh oh!

bors commented May 8, 2019

Uh oh!

bors commented May 8, 2019

Uh oh!

bjorn3May 8, 2019

Choose a reason for hiding this comment

Uh oh!

bjorn3May 8, 2019

Choose a reason for hiding this comment

Uh oh!

bjorn3 commented May 8, 2019

Uh oh!

Zoxc commented May 8, 2019

Uh oh!

bjorn3 commented May 8, 2019

Uh oh!

Zoxc commented May 8, 2019

Uh oh!

bjorn3 commented May 8, 2019

Uh oh!

Zoxc commented May 14, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants