Allow for re-using monomorphizations in upstream crates. by michaelwoerister · Pull Request #48779 · rust-lang/rust

michaelwoerister · 2018-03-06T13:49:46Z

Followup to #48611. This implementation is pretty much finished modulo failing tests if there are any. Not quite ready for review yet though.

DESCRIPTION

This PR introduces a share-generics mode for RLIBs and Rust dylibs. When a crate is compiled in this mode, two things will happen:

before instantiating a monomorphization in the current crate, the compiler will look for that monomorphization in all upstream crates and link to it, if possible.
monomorphizations are not internalized during partitioning. Instead they are added to the list of symbols exported from the crate.

This results in less code being translated and LLVMed. However, there are also downsides:

it will impede optimization somewhat, since fewer functions can be internalized, and
Rust dylibs will have bigger symbol tables since they'll also export monomorphizations.

Consequently, this PR only enables the shared-generics mode for opt-levels No, Less, Size, and MinSize, and for when incremental compilation is activated. -O2 and -O3 will still generate generic functions per-crate.

Another thing to note is that this has a somewhat similar effect as MIR-only RLIBs, in that monomorphizations are shared, but it is less effective because it cannot share monomorphizations between sibling crates:

 A <--- defines `fn foo<T>() { .. }`
/ \
/ \
B C <--- both call `foo<u32>()`
\ /
\ /
D <--- calls `foo<u32>()` too

With share-generics, both B and C have to instantiate foo<u32> and only D can re-use it (from either B or C). With MIR-only RLIBs, B and C would not instantiate anything, and in D we would then only instantiate foo<u32> once.
On the other hand, when there are many leaf crates in the graph (e.g. when compiling many individual test binaries) then the share-generics approach will often be more effective.

TODO

Add codegen test that makes sure monomorphizations can be internalized in non-Rust binaries.
Add codegen-units test that makes sure we share generics.
Add run-make test that makes sure we don't export any monomorphizations from non-Rust binaries.
Review for reproducible-builds implications.

rust-highfive · 2018-03-06T13:49:48Z

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

michaelwoerister · 2018-03-06T14:05:35Z

@bors try

bors · 2018-03-06T14:05:45Z

⌛ Trying commit 35c9b1f2f0a365187789fc684cdb5eba9afb81dd with merge 55221f5e2d6d5c71f6b89674eb29f2a213f415ef...

bors · 2018-03-06T16:10:49Z

☀️ Test successful - status-travis
State: approved= try=True

michaelwoerister · 2018-03-06T20:04:21Z

@Mark-Simulacrum, could you do a perf run for this too, please?

Mark-Simulacrum · 2018-03-07T02:35:10Z

Perf queued. Probably about 40-45 minutes until it starts.

michaelwoerister · 2018-03-07T09:06:42Z

Thanks, @Mark-Simulacrum! Here's the link:
http://perf.rust-lang.org/compare.html?start=6f2100b92cb14fbea2102701af6a3ac5814bd06c&end=55221f5e2d6d5c71f6b89674eb29f2a213f415ef&stat=instructions%3Au
(doesn't seem to have run yet though)

michaelwoerister · 2018-03-07T12:07:32Z

@Mark-Simulacrum, the results don't seem to be available yet. Is it still in the queue or has something gone wrong?

Mark-Simulacrum · 2018-03-07T14:11:49Z

Hm, it does look like something went wrong -- I've restarted the build.

michaelwoerister · 2018-03-07T16:33:59Z

Will the link be the same?

Mark-Simulacrum · 2018-03-07T16:53:58Z

Hm, it failed again -- I'm going to try and keep an eye on it and hopefully diagnose why, it also turns out we weren't properly logging the failures for try builds previously so I've now corrected that as well.

Mark-Simulacrum · 2018-03-07T23:41:38Z

URL works now!

michaelwoerister · 2018-03-08T08:59:17Z

Thanks, Mark!

michaelwoerister · 2018-03-08T09:08:49Z

OK, so those numbers look good. I had hoped that they would be even better though. It looks like it's mostly small functions that get re-used. But yeah, -15.9% for tokio-web-push, I'll take it :)

michaelwoerister · 2018-03-08T09:47:02Z

@rust-lang/compiler & @alexcrichton, do you have any objections to pursuing this further? There's a description at the top and performance numbers are here: http://perf.rust-lang.org/compare.html?start=6f2100b92cb14fbea2102701af6a3ac5814bd06c&end=55221f5e2d6d5c71f6b89674eb29f2a213f415ef&stat=instructions%3Au

alexcrichton · 2018-03-08T15:47:34Z

Awesome work here @michaelwoerister! It's pretty neat how it's not to difficult to play around with various schemes like this these days :)

One concern I might have here is the size of binaries but given that this only affects debug mode rather than optimized then I guess it doesn't matter too much? We rely on -ffunction-sections to pretty aggressively prune dead code but if the symbols are all exported then I think even for binaries they'll stick around?

In general though seems like a great idea to me to keep pursuing, any bugs or surprises along the way we can probably smooth over!

For the diamond problem you gisted above, is this what all that "link once ODR" stuff is for in LLVM? I feel like that's all basically intended for optimized binaries linking only one copy rather than for debug mode, so it may not benefit us much if we don't turn this on in optimized mode. Speaking of optimized mode though, we may actually be able to get some nice wins here with available_externally in LLVM. That'd allow optimized mode to inline all the upstream copies but we get to completely skip codegen for them. Perhaps something to pursue down the road!

michaelwoerister · 2018-03-08T16:07:33Z

We rely on -ffunction-sections to pretty aggressively prune dead code but if the symbols are all exported then I think even for binaries they'll stick around?

The monomorphizations are still assigned SymbolExportLevel::Rust, so for executables, cdylibs, and staticlibs, the linker script should hide them. I'll better add a test for that though.

We could look into LinkOnceODR but I remember it causing problems for MingGW. It would only reduce the size of linked artifacts, not compile times though, I think, as we'd still have to translate and optimize twice in the example above. Otoh, it might make symbol naming a bit less complicated.

michaelwoerister · 2018-03-08T16:08:09Z

Also, thanks for the feedback, @alexcrichton! :)

alexcrichton · 2018-03-08T16:10:31Z

Oh right that's true, I'd sort of doubt that LinkOnceODR is portable enough for us to effectively leverage it...

I think that for executables we don't currently pass linker scripts/symbol whitelists, but AFAIK that's because we just never have before. We could likely start now!

michaelwoerister · 2018-03-08T16:19:16Z

I think that for executables we don't currently pass linker scripts/symbol whitelists

OK, I'll make sure we do as part of the PR.

nikomatsakis · 2018-03-08T16:36:14Z

We discussed this in the @rust-lang/compiler meeting today. Everybody felt pretty good about it. It'd be nice to land this and possibly do further experimentation to see if we can enable in optimized builds without hurting perf.

bors · 2018-04-05T16:38:11Z

💔 Test failed - status-travis

TimNN · 2018-04-05T16:38:13Z

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:51:32] error: linking with `x86_64-unknown-freebsd10-clang` failed: exit code: 1
[00:51:32] |
[00:51:32] = note: "x86_64-unknown-freebsd10-clang" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.rustc0-917b9ef0a7b983cf10797274482a861d.rs.rcgu.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.rustc1-917b9ef0a7b983cf10797274482a861d.rs.rcgu.o" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae" "-Wl,--version-script=/tmp/rustc.ovi2X4I7jAUg/list" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.crate.allocator.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/backtrace-sys-bbe90d914b660380/out/.libs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/miniz-sys-713bb475660d2297/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_driver-318a3acf8bf109be" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_trans_utils-001c5f6b2c92323d" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_traits-abdbf0712f2401b0" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_save_analysis-53e40058076da0ef" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_resolve-8082fbc965be0986" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_incremental-134d8bde5319837c" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_privacy-41b52ed52133044e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_typeck-396f6215599370cf" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_platform_intrinsics-cb9a1c5377e47375" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_plugin-f38fc8dfaf8b6760" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_metadata-16e1947c3e346b2c" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax_ext-f77da4db9e699c55" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_lint-fb635406d58e190a" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_passes-e9b4633f0391ae84" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_borrowck-c1a15b35dd0232af" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_mir-1a76dc326d635198" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_allocator-8f934385ab076dce" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc-b8acf2095ed8292e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "test-6998d1bf7de4cf82" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "term-bf017472caffaa64" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "proc_macro-fc11c98f53e80a0d" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_const_math-c0d8ab93c9398803" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_back-e7310054ad1e00b3" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax-a15c364b4de9175e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_errors-f27a08473fd3fcf1" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax_pos-683294ae62852d4b" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_data_structures-3757b603de6a4cc9" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "serialize-a300abde726fce3e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_cratesio_shim-91609fa4acae312f" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "fmt_macros-a950aec3d66841d9" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "graphviz-c97014d02962e350" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "arena-4edac79a5a35263e" "-Wl,--start-group" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "std-618f925df1ad5afc" "-Wl,--end-group" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib/libcompiler_builtins-2a783bc37f13b02d.rlib" "-Wl,-Bdynamic" "-l" "util" "-l" "execinfo" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-Wl,-rpath,$ORIGIN/../lib"
[00:51:32] = note: /usr/local/x86_64-unknown-freebsd10/bin/ld: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae: local symbol `__progname' in /usr/local/x86_64-unknown-freebsd10/usr/lib/Scrt1.o is referenced by DSO
[00:51:32] /usr/local/x86_64-unknown-freebsd10/bin/ld: final link failed: Bad value
[00:51:32] clang: error: linker command failed with exit code 1 (use -v to see invocation)
---
[00:51:32] process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name rustc rustc/rustc.rs --color always --error-format json --crate-type bin --emit=dep-info,link -C opt-level=2 --cfg feature="jemalloc" --cfg feature="rustc_back" -C metadata=507a1d01d9d54bae -C extra-filename=-507a1d01d9d54bae --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps --target x86_64-unknown-freebsd -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps --extern rustc_back=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/librustc_back-e7310054ad1e00b3.so --extern rustc_driver=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/librustc_driver-318a3acf8bf109be.so -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/backtrace-sys-bbe90d914b660380/out/.libs -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/miniz-sys-713bb475660d2297/out` (exit code: 101)
[00:51:32] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-freebsd" "-j" "4" "--release" "--locked" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:51:32] expected success, got: exit code: 101
[00:51:32] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1085:9
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:02a31f79:start=1522946225071277773,finish=1522946225079821480,duration=8543707
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:05fe9098
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:05fe9098:start=1522946225086324441,finish=1522946225091339815,duration=5015374
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0113ba3a
$ dmesg | grep -i kill
[ 10.179776] init: failsafe main process (1095) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

TimNN · 2018-04-05T16:38:16Z

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:51:32] error: linking with `x86_64-unknown-freebsd10-clang` failed: exit code: 1
[00:51:32] |
[00:51:32] = note: "x86_64-unknown-freebsd10-clang" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.rustc0-917b9ef0a7b983cf10797274482a861d.rs.rcgu.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.rustc1-917b9ef0a7b983cf10797274482a861d.rs.rcgu.o" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae" "-Wl,--version-script=/tmp/rustc.ovi2X4I7jAUg/list" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae.crate.allocator.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/backtrace-sys-bbe90d914b660380/out/.libs" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/miniz-sys-713bb475660d2297/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_driver-318a3acf8bf109be" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_trans_utils-001c5f6b2c92323d" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_traits-abdbf0712f2401b0" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_save_analysis-53e40058076da0ef" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_resolve-8082fbc965be0986" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_incremental-134d8bde5319837c" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_privacy-41b52ed52133044e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_typeck-396f6215599370cf" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_platform_intrinsics-cb9a1c5377e47375" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_plugin-f38fc8dfaf8b6760" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_metadata-16e1947c3e346b2c" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax_ext-f77da4db9e699c55" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_lint-fb635406d58e190a" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_passes-e9b4633f0391ae84" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_borrowck-c1a15b35dd0232af" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_mir-1a76dc326d635198" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_allocator-8f934385ab076dce" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc-b8acf2095ed8292e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "test-6998d1bf7de4cf82" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "term-bf017472caffaa64" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "proc_macro-fc11c98f53e80a0d" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_const_math-c0d8ab93c9398803" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_back-e7310054ad1e00b3" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax-a15c364b4de9175e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_errors-f27a08473fd3fcf1" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "syntax_pos-683294ae62852d4b" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_data_structures-3757b603de6a4cc9" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "serialize-a300abde726fce3e" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "rustc_cratesio_shim-91609fa4acae312f" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "fmt_macros-a950aec3d66841d9" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "graphviz-c97014d02962e350" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps" "-l" "arena-4edac79a5a35263e" "-Wl,--start-group" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib" "-l" "std-618f925df1ad5afc" "-Wl,--end-group" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-freebsd/lib/libcompiler_builtins-2a783bc37f13b02d.rlib" "-Wl,-Bdynamic" "-l" "util" "-l" "execinfo" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-Wl,-rpath,$ORIGIN/../lib"
[00:51:32] = note: /usr/local/x86_64-unknown-freebsd10/bin/ld: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/rustc-507a1d01d9d54bae: local symbol `__progname' in /usr/local/x86_64-unknown-freebsd10/usr/lib/Scrt1.o is referenced by DSO
[00:51:32] /usr/local/x86_64-unknown-freebsd10/bin/ld: final link failed: Bad value
[00:51:32] clang: error: linker command failed with exit code 1 (use -v to see invocation)
---
[00:51:32] process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name rustc rustc/rustc.rs --color always --error-format json --crate-type bin --emit=dep-info,link -C opt-level=2 --cfg feature="jemalloc" --cfg feature="rustc_back" -C metadata=507a1d01d9d54bae -C extra-filename=-507a1d01d9d54bae --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps --target x86_64-unknown-freebsd -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps --extern rustc_back=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/librustc_back-e7310054ad1e00b3.so --extern rustc_driver=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/deps/librustc_driver-318a3acf8bf109be.so -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/backtrace-sys-bbe90d914b660380/out/.libs -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-freebsd/release/build/miniz-sys-713bb475660d2297/out` (exit code: 101)
[00:51:32] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-freebsd" "-j" "4" "--release" "--locked" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:51:32] expected success, got: exit code: 101
[00:51:32] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1085:9
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:02a31f79:start=1522946225071277773,finish=1522946225079821480,duration=8543707
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:05fe9098
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:05fe9098:start=1522946225086324441,finish=1522946225091339815,duration=5015374
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0113ba3a
$ dmesg | grep -i kill
[ 10.179776] init: failsafe main process (1095) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

…hare-generics

…upport Rust dylibs.

michaelwoerister · 2018-04-06T12:08:05Z

@bors r=alexcrichton

bors · 2018-04-06T12:08:06Z

📌 Commit 61991a5 has been approved by alexcrichton

bors · 2018-04-06T15:01:34Z

⌛ Testing commit 61991a5 with merge 7678d50...

bors · 2018-04-06T17:38:37Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 7678d50 to master...

eddyb · 2019-04-11T06:49:26Z

 pub const TAG_INVALID_SPAN: u8 = 1;
+
+#[derive(RustcEncodable, RustcDecodable)]
+pub struct EncodedExportedSymbols {


This should have a comment on it, that it's used to avoid adding a 'tcx parameter to CrateRoot (which I'm not even sure is a problem, if covariant, we'd just store it as 'static).

rust-highfive assigned estebank Mar 6, 2018

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 6, 2018

michaelwoerister unassigned estebank Mar 6, 2018

michaelwoerister force-pushed the share-generics4 branch from 9a1af56 to 710e4d6Compare March 7, 2018 12:57

michaelwoerister mentioned this pull request Mar 8, 2018
Compiler Performance Tracking Issue #48547
Closed

michaelwoerister force-pushed the share-generics4 branch 2 times, most recently from be1e8f6 to d4264dcCompare March 13, 2018 13:50

michaelwoerister changed the title ~~WIP: Allow for re-using monomorphizations in upstream crates.~~Allow for re-using monomorphizations in upstream crates.Mar 13, 2018

michaelwoerister added 16 commits April 6, 2018 12:14

Fix incomplete ICH implementation for ty::subst::UnpackedKind.
d3b5451

Allow for representing exported monomorphizations in crate metadata.
dad194a

Move export level computation to reachable_non_generics query.
435477d

Allow for re-using monomorphizations from upstream crates.
4f6d05d

Make generics sharing the default for non-optimized builds.
8d95c86

Remove the (inaccurate) symbol_export_level query.
e203b3a

Allow for internalizing monomorphizations that cannot be shared.
9b90674

Adapt codegen-unit test to shared-generics.
5316a45

Select upstream monomorphizations in a stable way.
213ef11

Fix some rebasing fallout.
a1a986c

Don't internalize generics that are re-exported
2d2cf03

Make sure that generics are internalized in executables even with -Zs…
94d36cf
…hare-generics

Add codegen-units test for shared-generics.
69c7f5c

Update a few comments about symbol visibility.
ec55390

Allow for re-using hidden monomorphizations on platforms that don't s…
07704a4
…upport Rust dylibs.

Update run-make/symbol-visibility to also cover shared-generics
61991a5

bors mentioned this pull request Apr 6, 2018
Introduce RangeInclusive::{new, start, end} methods and make the fields private. #49724
Merged

hanna-kruppe mentioned this pull request Jul 25, 2018
Make Vec derefing inlinable #52704
Closed

michaelwoerister mentioned this pull request Apr 2, 2019
Experiment with sharing monomorphized code between crates #47317
Closed

eddyb reviewed Apr 11, 2019
View reviewed changes

ehuss mentioned this pull request Sep 9, 2019
Using profile-overrides results in both optimized and unoptimized versions of the same crate in linked executable #63484
Closed

alessandrod mentioned this pull request Feb 11, 2022
Reduce compiled program sizes by reducing serialization bloat solana-labs/solana#23075
Closed

EFanZh mentioned this pull request Jul 17, 2023
Binary size optimization experiment rust-lang/log#569
Closed

Uh oh!

Uh oh!

Conversation

michaelwoerister commented Mar 6, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

DESCRIPTION

TODO

Uh oh!

rust-highfive commented Mar 6, 2018

Uh oh!

michaelwoerister commented Mar 6, 2018

Uh oh!

bors commented Mar 6, 2018

Uh oh!

bors commented Mar 6, 2018

Uh oh!

michaelwoerister commented Mar 6, 2018

Uh oh!

Mark-Simulacrum commented Mar 7, 2018

Uh oh!

michaelwoerister commented Mar 7, 2018

Uh oh!

michaelwoerister commented Mar 7, 2018

Uh oh!

Mark-Simulacrum commented Mar 7, 2018

Uh oh!

michaelwoerister commented Mar 7, 2018

Uh oh!

Mark-Simulacrum commented Mar 7, 2018

Uh oh!

Mark-Simulacrum commented Mar 7, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

alexcrichton commented Mar 8, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

alexcrichton commented Mar 8, 2018

Uh oh!

michaelwoerister commented Mar 8, 2018

Uh oh!

nikomatsakis commented Mar 8, 2018

Uh oh!

bors commented Apr 5, 2018

Uh oh!

TimNN commented Apr 5, 2018

Uh oh!

TimNN commented Apr 5, 2018

Uh oh!

michaelwoerister commented Apr 6, 2018

Uh oh!

bors commented Apr 6, 2018

Uh oh!

bors commented Apr 6, 2018

Uh oh!

bors commented Apr 6, 2018

Uh oh!

eddybApr 11, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

michaelwoerister commented Mar 6, 2018•
edited
Loading