add support for FreeBSD cross-compilation by wfurt · Pull Request #34000 · dotnet/runtime

wfurt · 2020-03-24T04:52:34Z

This change allows to cross-compile FreeBSD on Linux using ./build.sh -cross -os FreeBSD
(depends on dotnet/arcade#5114 and dotnet/arcade#5100)

In general, this is using existing cross-compilation support. However, when we cross-compile for different CPU architecture, we can use host and target somewhat interchangeably. As we build arm Linux on x64 Linux) With this we need cleaner (and correct) separation. With the example above, __HostOS and CLR_CMAKE_HOST_OS is set to Linux while __TargetOS and CLR_CMAKE_TARGET_OS is set to FreeBSD. With that in mind, CLR_CMAKE_HOST_LINUX should not be used to determine target options as well as created binaries are not runnable on host even if CPU architecture is same.

-fuse-ld=lld is no longer needed with clang90 - both native and cross-compiled build.

So far there is no CI. I did verify that generated pal/src/config.h is identical with one produced by running native build on FreeBSD.

…sdCross1

sdmaclea

I think there is a misunderstanding on the intended purpose of Host and Target.

We are using the gcc/gdb/llvm/lldb model for the definitions of Build, Host, Target.

Build the OS that the compiler is running on.
Host the OS of the machine where the compiled code will run.
Target the machine the which the running code will compile/debug.

In most cases TargetOS == HostOS. We have a few components which run on a different machine than their target. So far these are code generation and debug tools. For example we have a crossgen which can run on x64, but generate code for Arm64. I am also working on debugger tools which will run on Windows but debug a Linux runtime.

I think you are just trying to enable cross compilation of the runtime on Linux targeting FreeBSD. In this case the HostOS and the TargetOS should both be set to FreeBSD.

wfurt · 2020-03-25T17:25:36Z

We are using the gcc/gdb/llvm/lldb model for the definitions of Build, Host, Target.

Where is that definition? I only see use of Host and Target OS and ARCH through out the project files. And we already for example build Linux on Windows.

wfurt · 2020-03-25T17:39:15Z

I check and for the `linuxdac' target on Windows and I see CLR_CMAKE_HOST_OS=Windows_NT and CLR_CMAKE_TARGET_OS=Linux. (CLR_CMAKE_BUILD_OS is not set)

sdmaclea · 2020-03-25T17:44:26Z

I check and for the `linuxdac' target on Windows and I see CLR_CMAKE_HOST_OS=Windows_NT and CLR_CMAKE_TARGET_OS=Linux. (CLR_CMAKE_BUILD_OS is not set)

Yes. That command line option builds a DAC on a Windows build machine that runs on a Windows host to debug a Linux target runtime.

There is very little mention of Build in the make system so far. In general, the Build machine has made little difference.

/cc @jkotas.

sdmaclea · 2020-03-25T17:46:38Z

Where is that definition?

see
#1887 (comment)

sdmaclea · 2020-03-25T17:52:25Z

One source of confusion is that by default Build machine properties sets the Host variables.... This is due to the simple assumption that in many cases Build == Host == Target.

…sdCross2

wfurt · 2020-03-26T21:46:56Z

This should be hopefully ready for final pass. I made change @jkotas suggested to always pass -os the the shell script. I also did basic verification that produced binaries do run (on FreeBSD 12.0p9)

[furt@daemon /tmp/dotnet]$ ./dotnet --info
Host (useful for support):
Version: 5.0.0-dev
Commit: a288211ca0
.NET Core SDKs installed:
No SDKs were found.
.NET Core runtimes installed:
Microsoft.NETCore.App 5.0.0-dev [/tmp/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
[furt@daemon /tmp/dotnet]$ ./dotnet --info
Host (useful for support):
Version: 5.0.0-dev
Commit: a288211ca0
.NET Core SDKs installed:
No SDKs were found.
.NET Core runtimes installed:
Microsoft.NETCore.App 5.0.0-dev [/tmp/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download

akoeplinger · 2020-03-27T11:12:41Z

@@ -1,4 +1,5 @@
 set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
+set(CROSS_ROOTFS $ENV{ROOTFS_DIR})


isn't this already set by eng/common/cross/toolchain.cmake ?

That does not seems to be set soon enough and build fails. src/coreclr/tryrun.cmake has same line.

…sdCross2

am11 · 2020-03-31T09:58:11Z

@wfurt, @janvorli, a related question:

In order to support non-Linux/macOS Unix, the requirements are:

build the runtime native components targeting the OS of interest (cross compile or on-device)
build the managed components using crossgen with RID=<TargetOSName>-<TargetOSArch> on a supported OS like Ubuntu, so the TargetOS-specific managed partial classes get compiled correctly.
acquire dotnet SDK tarball for Linux (matching the exact git commit SHA-1)
replace native + managed assemblies from artifacts/ directory to dotnet. This should give us the bootstrap cli, right?
using this (bootstrap, stage-0) CLI, build the source-build tarball

With this PR changes, would we be able to get #1 and #2 using:

./build.sh -cross -os FreeBSD -configuration Release

such that the produced artifacts/ directory has everything ready for drop-in replacement for step #4?

…sdCross2

wfurt · 2020-03-31T20:36:28Z

perf leg failure seems unrelated.

wfurt · 2020-03-31T20:41:45Z

such that the produced artifacts/ directory has everything ready for drop-in replacement for step #4?

yes, I think so.
In my (debug) case I used artifacts/packages/Debug/Shipping/dotnet-runtime-5.0.0-dev-freebsd-x64.tar.gz. That should be complete drop AFAIK.

for #3 all you needs to do is to avoid cross-genned or use environmental variables to suppress R2R.

I was planning to feed cross-compiled runtime + SDK back to source-build. That may be good test.

ericstj · 2020-04-01T05:58:10Z

This change appears to be causing failures in all PRs due to passing -os in cases where it isn't supported.

am11 · 2020-04-01T06:29:22Z

Looks like Build Windows_NT x86 release Runtime_Debug leg which also failed to build in this PR. src/coreclr/build-runtime.sh supports -os, but src/coreclr/build-runtime.cmd is missing it. I think the fix would be to add an -os option at

runtime/src/coreclr/build-runtime.cmd

Line 150 in 2ad6913

if /i "%1"=="-alpinedac" (set __BuildNative=0&set __BuildCrossArchNative=1&set __CrossArch=x64&set __TargetOS=alpine&set processedArgs=!processedArgs!%1&shift&goto Arg_Loop) 

jkotas · 2020-04-01T08:45:01Z

perf leg failure seems unrelated.

@wfurt The link to the issue tracking the unrelated failure should be added next time. https://github.com/dotnet/runtime/blob/master/docs/pr-guide.md#what-to-do-if-you-determine-the-failure-is-unrelated

jaredpar · 2020-04-01T15:19:59Z

runtime-live-build (Build Windows_NT x86 release Runtime_Debug) Build Wi

This leg was also failing and was not addressed in the justification for merging.

add support for FreeBSD cross-compilation
cbc70ea

wfurt added os-freebsd FreeBSD OS area-Infrastructure labels Mar 24, 2020

wfurt requested review from ViktorHofer, janvorli and safern March 24, 2020 04:52

wfurt self-assigned this Mar 24, 2020

am11 reviewed Mar 24, 2020
View reviewed changes

Comment threadeng/native/init-distro-rid.sh

am11 reviewed Mar 24, 2020
View reviewed changes

Comment threadeng/native/init-distro-rid.sh Outdated

am11 reviewed Mar 24, 2020
View reviewed changes

Comment threadsrc/coreclr/src/pal/src/configure.cmake Outdated

am11 reviewed Mar 24, 2020
View reviewed changes

Comment threaddocs/workflow/building/libraries/freebsd-instructions.md Outdated

wfurt added 4 commits March 24, 2020 19:56

feedback from review
6c1904b

update detection
c99ead5

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto freeb…
0ac49a7
…sdCross1

add openssl to docs
a288211

janvorli reviewed Mar 25, 2020
View reviewed changes

Comment threadsrc/coreclr/src/dlls/mscordac/CMakeLists.txt Outdated
Comment threadsrc/coreclr/src/pal/src/CMakeLists.txt
Comment threadsrc/coreclr/src/pal/src/configure.cmake

wfurt requested a review from sdmaclea March 25, 2020 02:26

sdmaclea previously requested changes Mar 25, 2020
View reviewed changes

jaredpar mentioned this pull request Mar 25, 2020
Test Failed: GetAsync_SetCookieContainer_CookieSent failed #12043
Closed

wfurt added 3 commits March 26, 2020 06:14

feedback from reviews
aca7dd4

feedback from reviews
1d094eb

final cleanup
b47bdbc

jkotas reviewed Mar 26, 2020
View reviewed changes

Comment threadsrc/coreclr/crossgen-corelib.proj Outdated

jkotas reviewed Mar 26, 2020
View reviewed changes

Comment threadsrc/coreclr/runtime.proj Outdated

jkotas reviewed Mar 26, 2020
View reviewed changes

Comment threadsrc/installer/corehost/build.proj Outdated

sdmaclea reviewed Mar 26, 2020
View reviewed changes

Comment threadsrc/coreclr/crossgen-corelib.sh Outdated

wfurt added 2 commits March 26, 2020 18:25

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto freeb…
abc273b
…sdCross2

feedback from reviews
df82e75

sdmaclea approved these changes Mar 26, 2020
View reviewed changes

akoeplinger reviewed Mar 27, 2020
View reviewed changes

wfurt added 4 commits March 30, 2020 13:52

simplify detection of route.h
3dc12ec

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto freeb…
ec1cdbd
…sdCross2

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto freeb…
8ad20d6
…sdCross2

correct __CrossBuild detection
e06741a

jkotas reviewed Mar 31, 2020
View reviewed changes

Comment threadsrc/installer/Directory.Build.props Outdated

wfurt added 2 commits March 31, 2020 10:08

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto freeb…
090bf2d
…sdCross2

use TargetOS for installer
e8d2a20

jkotas approved these changes Mar 31, 2020
View reviewed changes

wfurt merged commit 595a95c into dotnet:masterMar 31, 2020

wfurt deleted the freebsdCross2 branch March 31, 2020 20:36

wfurt mentioned this pull request Mar 31, 2020
Port corehost to QNX7 #33374
Open

ericstj mentioned this pull request Apr 1, 2020
Fix System.Data.SqlClient baseline #34352
Merged

safern mentioned this pull request Apr 1, 2020
Fix build break in coreclr where -os is not supported in Windows #34358
Merged

am11 mentioned this pull request Apr 20, 2020
Building the .NET Core SDK on FreeBSD dotnet/source-build#1139
Closed

ghost locked as resolved and limited conversation to collaborators Dec 10, 2020

		@@ -1,4 +1,5 @@
		set(TARGET_ARCH_NAME $ENV{TARGET_BUILD_ARCH})
		set(CROSS_ROOTFS $ENV{ROOTFS_DIR})

Uh oh!

Conversation

wfurt commented Mar 24, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdmaclea left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wfurt commented Mar 25, 2020

Uh oh!

wfurt commented Mar 25, 2020

Uh oh!

sdmaclea commented Mar 25, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sdmaclea commented Mar 25, 2020

Uh oh!

sdmaclea commented Mar 25, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wfurt commented Mar 26, 2020

Uh oh!

Uh oh!

akoeplingerMar 27, 2020

Choose a reason for hiding this comment

Uh oh!

wfurtMar 30, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

am11 commented Mar 31, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wfurt commented Mar 31, 2020

Uh oh!

wfurt commented Mar 31, 2020

Uh oh!

ericstj commented Apr 1, 2020

Uh oh!

am11 commented Apr 1, 2020

Uh oh!

jkotas commented Apr 1, 2020

Uh oh!

jaredpar commented Apr 1, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

sdmaclea commented Mar 25, 2020•
edited
Loading

am11 commented Mar 31, 2020•
edited
Loading