Build on CBL-mariner host with rootfs by sbomer · Pull Request #84148 · dotnet/runtime

sbomer · 2023-03-30T21:30:36Z

This changes the linux builds to use the CBL-mariner build images added in dotnet/dotnet-buildtools-prereqs-docker#832. All of these builds are now cross-builds with a rootfs (including x64, and x64 musl).

The new images intentionally don't have binutils, so our build jobs have been updated to use llvm-based tools. This includes lld when we run nativeaot over our tests. This leads to a slight size regression, I believe due to ILCompiler not supporting --gc-sections with lld (see Add LinkerFlavor to NativeAOT #83558 (comment)). cc @MichalStrehovsky
This change turns off PGO optimization because it was hitting a compiler crash when consuming and old version of profile data produced by clang9 instrumented builds (https://bugzilla.redhat.com/show_bug.cgi?id=1827282). The intention is to quickly flow the change into our optimization repo, then collect and ingest the new data so we can turn optimization back on.
The Mono LLVMAot test build is kept on the old build images, because they run the aot compiler which depends on binutils. We can probably move it to the same plan as the other legs, but I'm not familiar with the context here. cc @akoeplinger. We would at least need to fix AS_NAME here:
runtime/src/mono/mono/mini/aot-compiler.c
Line 13030 in 1d2cd20
#defineAS_NAME "as"
Mono arm64 product build is kept on the old images due to Unable to compile mono arm64 with clang12 #84503.

Contributes to #83428.

This change enables cross-building for Alpine x64 on other x64 distro.

Necessary because pkg-config on mariner is a shim for pkgconf, which doesn't support override variables.

- Use lld for LTO instead of ld.gold - Set cross toolchain options for x64 mono build - Prefer llvm-objcopy for mono symbol stripping

Our custom LLVM build seems to have different defaults when targeting ARM than the clang-12 available in ubuntu. Probably related to https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/commit/b7363248b115339c4fb838fcd3ae43671eedae0a. Passing the -march explicitly should not hurt.

Allow text relocations. The lld defaults differ from ld, which will add text relocations on demand by default. See https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities

+ more cleanup

To locate gcc toolchain in target sysroot, when host arch matches target arch.

ghost · 2023-03-30T21:30:46Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

null

Author:	sbomer
Assignees:	sbomer
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

@BruceForstall

Fixesdotnet/runtime#84780. Tested by hooking up the changes locally with runtime's `jitformat.py` and running `python3 ./src/coreclr/scripts/jitformat.py -c /runtime/src/coreclr -o linux -a x64` in our mariner build container. This might break scenarios where jitutils are used outside of ci on a non-mariner build image. Per discussion with @BruceForstall let's make a quick fix to unblock ci, and if this is problematic we could for example add an extra argument that controls whether the `-cross` argument is passed to the runtime build. For context, the `-cross` argument is necessary after dotnet/runtime#84148 because our linux build images have change from centos 7 to CBL-mariner, and the linux builds are now cross-builds that use a rootfs (even x64 and x64-musl).

rzikm · 2023-04-19T07:55:40Z

This PR (or a related one) breaks System.Net.Security.Enterprise.Test builds (https://dev.azure.com/dnceng-public/public/_build/results?buildId=243440&view=logs&j=03b95830-93c2-599b-7e4d-1efb940adc28&t=9abf97cd-b4a3-5a26-6cf2-d05ce865f2ef&l=3907).

clang : error : invalid linker name in argument '-fuse-ld=lld' [/repo/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj]

The docker image used (mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04) has clang-9 but no lld binary installed. Will that be fixed by the TODO at
https://github.com/sbomer/runtime/blob/92d1f2217649103e3f3e5121b8ad35dca5a81bbb/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj#L62? Or is some additional detection needed?

To reflect the plan in dotnet/runtime#83428, and changes made in dotnet/runtime#84148.

janvorliand others added 20 commits February 9, 2023 11:19

Enable x64 Alpine crossbuild
8735c94
This change enables cross-building for Alpine x64 on other x64 distro.

Set PKG_CONFIG_SYSROOT_DIR
0084966
Necessary because pkg-config on mariner is a shim for pkgconf, which doesn't support override variables.

More build fixes
bb39763
- Use lld for LTO instead of ld.gold - Set cross toolchain options for x64 mono build - Prefer llvm-objcopy for mono symbol stripping

test
a5f2dbd

Print full clang command to show selected linker
db7bf3e

Build scripts
597a304

Fix x64 -> x64 musl cross-build
3e5a366

Allow cross compiling for x86
612c686

Enable x86 build with lld
a9fd083
Allow text relocations. The lld defaults differ from ld, which will add text relocations on demand by default. See https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities

Fix x86 mono cross-build
d0a1560

Some cleanup
2abee9b

Add gcc-toolchain arg when aot-ing crossgen
54dc641
+ more cleanup

Merge remote-tracking branch 'origin/main' into marinerTest
2da99a9

Fix AOT compilation of ILCompiler and crossgen2
825a1d9

Only pass --gcc-toolchain where required
c7468f6
To locate gcc toolchain in target sysroot, when host arch matches target arch.

Cleanup
ae6c6ab

Fix symbol stripping on ci builds
3d70352

Fix nativeaot test build
2f12a9a

Pick up mariner images, set cross-build arguments
f6bd3e5

ghost added the area-Infrastructure-coreclr Only use for closed issues label Mar 30, 2023

ghost assigned sbomer Mar 30, 2023

sbomer mentioned this pull request Mar 30, 2023
Add 'su' to mariner images dotnet/dotnet-buildtools-prereqs-docker#835
Merged

build-analysisBot mentioned this pull request Mar 31, 2023
WasmTestOnBrowser-System.* test failures in CI #83655
Closed

am11 reviewed Mar 31, 2023
View reviewed changes

Comment threadeng/Subsets.props Outdated

am11 reviewed Mar 31, 2023
View reviewed changes

Comment threadeng/common/cross/build-rootfs.sh

am11 reviewed Mar 31, 2023
View reviewed changes

Comment threadsrc/coreclr/pgosupport.cmake Outdated

sbomerand others added 2 commits March 31, 2023 16:27

Don't use lld on macos
e9f17fb

Update src/coreclr/pgosupport.cmake
26621ab
Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>

This was referenced Apr 13, 2023
Decide on linker to use in runtime builds #84794
Closed
Fix enterprise linux build #84795
Merged
Add runtime build dependencies to ubuntu 22.04 image dotnet/dotnet-buildtools-prereqs-docker#843
Merged

eerhardt mentioned this pull request Apr 13, 2023
[main] Update dependencies from dotnet/runtime dotnet/aspnetcore#47693
Merged

sbomer mentioned this pull request Apr 13, 2023
Add Azure DevOps dependencies to mariner fpm image dotnet/dotnet-buildtools-prereqs-docker#844
Merged

This was referenced Apr 14, 2023
jitrolling build is broken #84822
Closed
jitformat failing on linux x64 #84780
Closed

sbomer mentioned this pull request Apr 14, 2023
Pass "-cross" arg on linux builds dotnet/jitutils#369
Merged

sbomer mentioned this pull request Apr 14, 2023
Adjust managed type system for new function pointer handling #84819
Merged

clamp03 mentioned this pull request Apr 17, 2023
[RISC-V] Disable corelib crossgen2 #84781
Merged

BruceForstall mentioned this pull request Apr 18, 2023
Update coredistools dotnet/jitutils#372
Closed

This was referenced Apr 19, 2023
Add TargetHostName to QuicConnection #84976
Merged
System.Net.Http.Enterprise.Tests build failing in CI #85035
Closed

sbomer mentioned this pull request Apr 20, 2023
dotnet --info hitting /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found on Latest net8.0 dotnet/sdk#31893
Closed

antonfirsov mentioned this pull request Apr 25, 2023
[HttpStress] [SslStress] Linux stress test builds are broken #85338
Closed

sbomer added a commit to dotnet/core that referenced this pull request May 2, 2023
Update .NET 8 libc compat (#8401)
5da7900
To reflect the plan in dotnet/runtime#83428, and changes made in dotnet/runtime#84148.

sbomer mentioned this pull request May 3, 2023
Run NativeAot tests on arm64 #82307
Merged

ghost locked as resolved and limited conversation to collaborators May 19, 2023

sbomer deleted the marinerBuild branch August 7, 2025 17:43

Navigation Menu

Uh oh!

Build on CBL-mariner host with rootfs - #84148

Build on CBL-mariner host with rootfs#84148
sbomer merged 88 commits into
dotnet:mainfrom
sbomer:marinerBuild

sbomer commented Mar 30, 2023•
edited
Loading

Uh oh!

ghost commented Mar 30, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rzikm commented Apr 19, 2023•
edited
Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

sbomer commented Mar 30, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Mar 30, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rzikm commented Apr 19, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sbomer commented Mar 30, 2023•
edited
Loading

rzikm commented Apr 19, 2023•
edited
Loading