build-std compatible profile(code coverage) support

[ldm0]

1. Remove the injected profiler_builtins crate (user-provided profiler runtime through -Zprofiler-runtime is still injected)
2. Compile LLVM's compiler-rt/profile in bootstrap and copy it to sysroot(just like sanitizers)

fixes#79401
fixesrust-lang/wg-cargo-std-aware#63
fixesrust-lang/wg-cargo-std-aware#68

cc @ehuss@bjorn3@jyn514

[rust-highfive]

r? @jyn514

(rust-highfive has picked a reviewer for you, use r? to override)

[bjorn3]

Wouldn't this make profiling for targets other than the host impossible with -Zbuild-std?

[ldm0]

Wouldn't this make profiling for targets other than the host impossible with -Zbuild-std?

Do you mean cross compiling with profiling enabled? I think that's possible since profiler runtime is provided in the target lib in sysroot.

This PR makes profiler-runtime to be provided in the same way as santizer-runtimes (#65241 could be referenced). Therefore, profilers can be used in almost the same way as sanitizers.

[bjorn3]

Do you mean cross compiling with profiling enabled?

Yes

I think that's possible since profiler runtime is provided in the target lib in sysroot.

What if you want to use -Zbuild-std because there is no sysroot for the target in the first place?

[ldm0]

I think that's possible since profiler runtime is provided in the target lib in sysroot.

What if you want to use -Zbuild-std because there is no sysroot for the target in the first place?

Currenly without the prebuilt static lib, it's not possible. We can copy the source code with rust-src though. But it would be awkward to compile a cpp based compiler-rt when building std, unnecessary dependencies like cmake or cc will be required.

[rustbot]

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

• Stabilizing library features
• Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
• Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
• Changing public documentation in ways that create new stability guarantees
• Changing observable runtime behavior of library APIs

[jyn514]

How is this PR related to #101009 ? Does it require that PR to merge before? Does it make sense for #101009 to land separately, or should these two be combined?

[ldm0]

How is this PR related to #101009 ? Does it require that PR to merge before? Does it make sense for #101009 to land separately, or should these two be combined?

#101009 is expected to be a harmless&simple improvement which I expect to merge before this PR.

But If this PR is merged before #101009 , #101009 is useless since profiler_builtins is removed.

[ehuss]

I'm a bit confused on how this solves the linked issues. It looks like this depends on having profiler_builtins in the sysroot, which I mentioned in #101009 (comment) is probably not the direction I would want to go. Am I misreading how this works?

[ldm0]

I'm a bit confused on how this solves the linked issues. It looks like this depends on having profiler_builtins in the sysroot, which I mentioned in #101009 (comment) is probably not the direction I would want to go. Am I misreading how this works?

This PR make profiler provided in the same way as sanitizers(prebuilt *.a). They are all compiler-rt in LLVM, if providing sanitizers with prebuilt static libs was acceptable before, providing profiler runtime as static lib should also be acceptable now I guess? Nobody want to run cmake while building a rust project.

Ref: #65241

[bjorn3]

They are all compiler-rt in LLVM, if providing sanitizers with prebuilt static libs was acceptable before, providing profiler runtime as static lib should also be acceptable now I guess?

Sanitizer support was introduced several years before -Zbuild-std. If -Zbuild-std support had been introduced first, it would likely not have been considered acceptable to use precompiled static libs for sanitizers.

Nobody want to run cmake while building a rust project.

We already require a C compiler for building compiler-builtins on many targets. Is cmake that much worse? Would it be possible to rewrite the build logic for the parts of compiler-rt that we need in rust instead if this is the case?

[ldm0]

They are all compiler-rt in LLVM, if providing sanitizers with prebuilt static libs was acceptable before, providing profiler runtime as static lib should also be acceptable now I guess?

Sanitizer support was introduced several years before -Zbuild-std. If -Zbuild-std support had been introduced first, it would likely not have been considered acceptable to use precompiled static libs for sanitizers.

Profiler support was also introduced several years before -Zbuild-std(#42433).
If providing libunwind and sanitizers as static libs in sysroot is acceptable (#65241, #85600), this PR should also be acceptable.

Is cmake that much worse?

Add unnecessary CMake+Cpp dependencies in rust project is practically bad (various compilation issues will occurs) e.g. tokio-rs/prost#657

Would it be possible to rewrite the build logic for the parts of compiler-rt that we need in rust instead if this is the case?

compiler-rt has more than 10 thousand lines of CMake, porting them to Rust doesn't sounds like a good idea(error prone, LLVM upgrading burden, LLVM version locking)

Summary of my concerns:

1. rust doesn't vendor CMake yet. Introducing dependeny of CMake provided by user to avoid vendored dependencies(libclang_rt.{}_{}.a) might introduce more problems.
2. profiler_builtins cannot be a dependency of libcore as long as it's a crate with build script needed due to cyclic dependency problem(even if core is only a build-dependency). Providing profiler runtime as static lib effectivly resolves the problem.

Our company project is relying on -Zbuild-std due to enabling of -Zpanic-in-drop=abort, and PGO and instrument-coverage support are all broken by it currently. I would like to argue whether "This PR is not the solution what we ultimately want to have" is a strong enough argument to reject a legit bugfix.

[tmiasko]

What use-cases are you trying to address by suggesting this direction @bjorn3?

Using -Zbuild-std with a target other than the host. In that case there is no prebuilt sanitizer or profiler runtime availabe in the sysroot.

As mentioned earlier in this thread. It is already possible, but requires target to be installed.

[bjorn3]

That doesn't help anything for tier3 and external targets as they can't be installed.

[tmiasko]

I see, so you are interested in use-cases where those components have to be built because they are otherwise unavailable. I would expect that implementing support for such use-cases would be up to the people interested in them. I don't think we should block -Zbuild-std compatible implementation for that reason. Unless you have concrete proposal for alternative solution?

[bjorn3]

Maybe we could have a precompiled profiler runtime in the sysroot for tier 1 and tier 2 targets, but build it from scratch using cmake if the sysroot version is not available? This could be implemented by making the profiler_builtins crate #![no_std] and in the build script check if the .rlib is available for the target and if so link against it like a staticlib and if not build from scratch. This shouldn't need any rustbuild or cargo changes.

[bors]

☔ The latest upstream changes (presumably #101833) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

☔ The latest upstream changes (presumably #102975) made this pull request unmergeable. Please resolve the merge conflicts.

[jyn514]

Is this attribute still used anywhere? Can we remove it altogether?

[ldm0]

There is an unstable option "-Zprofiler-runtime=<crate_name>" for users to inject their custom profiler runtime. The profiler_runtime attribute is the profiler-runtime-crate marker. So if the attribute is removed, -Zprofiler-runtime will be useless.

[bors]

☔ The latest upstream changes (presumably #103345) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

☔ The latest upstream changes (presumably #103797) made this pull request unmergeable. Please resolve the merge conflicts.

[JohnCSimon]

@ldm0
ping from triage - can you post your status on this PR? Thanks

[JohnCSimon]

@ldm0
ping from triage - can you post your status on this PR? Thanks

[Dylan-DPC]

Closing this as inactive. Feel free to reöpen this pr or create a new pr if you get the time to work on this. Thanks