Only instantiate inline- and const-fns if they are referenced (again).

[michaelwoerister]

It seems that we have regressed on not translating #[inline] functions unless they are actually used. This should bring back this optimization. I also added a regression test this time so it doesn't happen again accidentally.

Fixes#40392.

r? @alexcrichton

UPDATE & PSA

This patch makes translation very lazy -- in general this is a good thing (we don't want the compiler to do unnecessary work) but it has two consequences:

1. Some error messages are only generated when an item is actually translated. Consequently, this patch will lead to more cases where the compiler will only start emitting errors when the erroneous function is actually used. This has always been true to some extend (e.g. when passing generic values to an intrinsic) but since this is something user-facing it's worth mentioning.
2. When writing tests, one has to make sure that the functions in question are actually generated. In other words, it must not be dead code. This can usually be achieved by either
   1. making sure the function is exported from the resulting binary or
   2. by making sure the function is called from something that is exported (or main()).

Note that it depends on the crate type what functions are exported:

1. For rlibs and dylibs everything that is reachable from the outside is exported.
2. For executables, cdylibs, and staticlibs, items are only exported if they are additionally #[no_mangle] or have an #[export_name].

The commits in this PR contain many examples of how tests can be updated to comply to the new requirements.

[alexcrichton]

Looks great to me! r=me with passing tests

[kennytm]

CI is still failing. These tests compiled successfully while they should not:

[00:51:51] [compile-fail] compile-fail/E0534.rs
[00:51:51] [compile-fail] compile-fail/bad-intrinsic-monomorphization.rs
[00:51:51] [compile-fail] compile-fail/dupe-symbols-2.rs
[00:51:51] [compile-fail] compile-fail/invalid-inline.rs
[00:51:51] [compile-fail] compile-fail/issue-22638.rs
[00:51:51] [compile-fail] compile-fail/non-interger-atomic.rs

[alexcrichton]

This looks great to me, thanks @michaelwoerister! I'm a little hesitant to personally r+ due to the change in semantics here. We chatted on IRC and it sounded like you're going to wait for discussion amongst the @rust-lang/compiler team, so I'll tag the PR appropriately for that.

[eddyb]

Some error messages are only generated when an item is actually translated.

I should note that this mainly has to do with unstable features (e.g. direct uses of intrinsics).
AFAIK, there are only two other ways to get an error (not a warning) during codegen:

• a type with a size in bits that cannot be represented in i64 (LLVM limitation), or more generally, with a size in bytes that cannot be represented in the target platform's usize
• resource limits, typically recursion either during trait matching or the function monomorphizations themselves

We could turn these into warnings and generate runtime panics instead, in fact the warnings could be done ahead-of-time and distinct from codegen anyway.

[michaelwoerister]

We could turn these into warnings and generate runtime panics instead, in fact the warnings could be done ahead-of-time and distinct from codegen anyway.

Yes, I think this should be the preferred way to do it anyway.

What does the rest of @rust-lang/compiler say? Can we land this? I think the general goal of being as lazy as possible is worth pursuing.

[arielb1]

Shouldn't private no_mangle functions always be compiled in?

[nikomatsakis]

Seems like it would be good if that were true.

[arielb1]

Shouldn't we check the inline attribute for functions that aren't translated? I don't want people to create functions with an inline attribute that explode when a child crate finally instantiates them.

[arielb1]

I'm not sure this is a difference in kind, merely a difference in degree: even today you can write things that depend on reachability and blow up only when a downstream crate uses them, for example:

#![feature(core_intrinsics)]use std::intrinsics;#[inline(fail)]pubfnfoo<T>(_t:T){unsafe{
intrinsics::atomic_load(1as*const[u8;1048576]);}}#[inline(always)]pubfnbar(){unsafe{
intrinsics::atomic_load(1as*const[u8;1048576]);}}fnmain(){}

I still think we want to sanitize attributes during type-checking, but intrinsics (which are unstable and intentionally error on monomorphization time) and resource limits (which are always a problem) should be OK.

[michaelwoerister]

Discussed in the @rust-lang/compiler team meeting yesterday and the conclusion was to proceed with this. The main concern was that #[inline] attributes would not be validated for functions that are not called. This should be simple to fix and I opened #45738 to keep track of that.

@bors r=nikomatsakis

[bors]

📌 Commit 8d4c92f has been approved by nikomatsakis

[bors]

⌛ Testing commit 8d4c92f80d97bc0e42d935ce7d23f6e3083a4937 with merge df38d994283db81c5176dc109d1894cd5232625f...

[bors]

💔 Test failed - status-travis

[kennytm]

Test failed, test/mir-opt/match_false_edges/rustc.node17.SimplifyBranches-initial.before.mir does not exist:

[00:48:30] failures:
[00:48:30] [00:48:30] ---- [mir-opt] mir-opt/match_false_edges.rs stdout ----
[00:48:30] thread '[mir-opt] mir-opt/match_false_edges.rs' panicked at 'Output file `/checkout/obj/build/x86_64-unknown-linux-gnu/test/mir-opt/match_false_edges/rustc.node17.SimplifyBranches-initial.before.mir` from test does not exist', /checkout/src/tools/compiletest/src/runtest.rs:2290:12
[00:48:30] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:30]
[00:48:30] [00:48:30] failures:
[00:48:30] [mir-opt] mir-opt/match_false_edges.rs

[michaelwoerister]

So I think these new failures are because #45710 got merged in between. The custom allocator implementations in these test cases fail linking because (I assume) the respective functions are not instantiated because they are not considered exported? Does that sound right, @alexcrichton?

I don't quite understand where these functions are supposed to come from (__rg_alloc, __rg_oom, __rg_grow_in_place, etc).

[alexcrichton]

@michaelwoerister oh dear sorry about that!

The #[global_allocator] attribute is where they come from, the expand module will turn such a static into something like:

staticFOO:MyAllocator = ...;mod some_internal_name {#[std_special_symbol]#[no_mangle]fn__rg_alloc(...) -> ... {FOO.some_method(...)}
...
}

So the symbols are injected by the compiler and their attributes are generated here. IIRC the linkage = "external" puts them on the path towards reachability and then rustc_std_internal_symbol was a change to make them hidden visibility instead of public.

[michaelwoerister]

Ah ok, the collector does not see the functions generated in librustc_trans/allocator.rs, and since we are creating an executable, these seemingly unreferenced SymbolExportLevel::Rust functions will not be instantiated.

[michaelwoerister]

@alexcrichton Do you think that there's a case where a #[rustc_std_internal_symbol] should not be instantiated? If not I think this could be solved by treating such functions the same as regularly exported functions in the collector.

[alexcrichton]

@michaelwoerister nah I think everything tagged with that should be unconditionally instantiated. They're all tiny anyway and will get gc'd by the linker if they're not used.

[michaelwoerister]

@alexcrichton 👍

[michaelwoerister]

OK, travis looks good. Let's give it another try.

@bors r=nikomatsakis

[bors]

📌 Commit d141abf has been approved by nikomatsakis

[bors]

⌛ Testing commit d141abf with merge 6dc72035c92f954c39240fbe4eeec544ef159331...

[bors]

💔 Test failed - status-travis

[kennytm]

dist-i586-gnu-i686-musl failed at codegen/fastcall-inreg.rs.

Details

[00:51:57] ---- [codegen] codegen/fastcall-inreg.rs stdout ----
[00:51:57] [00:51:57] error: verification with 'FileCheck' failed
[00:51:57] status: exit code: 1
[00:51:57] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/build/bin/FileCheck" "--input-file" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/codegen/fastcall-inreg.ll" "/checkout/src/test/codegen/fastcall-inreg.rs"
[00:51:57] stdout:
[00:51:57] ------------------------------------------
[00:51:57] [00:51:57] ------------------------------------------
[00:51:57] stderr:
[00:51:57] ------------------------------------------
[00:51:57] /checkout/src/test/codegen/fastcall-inreg.rs:63:12: error: expected string not found in input
[00:51:57] // CHECK: @f1(i32 inreg %arg0, i32 inreg %arg1, i32 %arg2)
[00:51:57] ^
[00:51:57] /checkout/obj/build/x86_64-unknown-linux-gnu/test/codegen/fastcall-inreg.ll:1:1: note: scanning from here
[00:51:57] ; ModuleID = 'fastcall_inreg0-8787f43e282added376259c1adb08b80.rs'
[00:51:57] ^
[00:51:57] /checkout/obj/build/x86_64-unknown-linux-gnu/test/codegen/fastcall-inreg.ll:3:35: note: possible intended match here
[00:51:57] target datalayout = "e-m:e-p:32:32-f64:32:64-f80:32-n8:16:32-S128"
[00:51:57] ^
[00:51:57] [00:51:57] ------------------------------------------
[00:51:57] [00:51:57] thread '[codegen] codegen/fastcall-inreg.rs' panicked at 'explicit panic', /checkout/src/tools/compiletest/src/runtest.rs:2498:8
[00:51:57] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:51:57] [00:51:57] [00:51:57] failures:
[00:51:57] [codegen] codegen/fastcall-inreg.rs
[00:51:57] [00:51:57] test result: �[31mFAILED�(B�[m. 47 passed; 1 failed; 5 ignored; 0 measured; 0 filtered out

[michaelwoerister]

Updated test case.

@bors r=nikomatsakis

[bors]

📌 Commit 081ef8e has been approved by nikomatsakis

[bors]

⌛ Testing commit 081ef8e with merge 4bb96f6...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing 4bb96f6 to master...

[michaelwoerister]

Wow, cool :)