Beginnings of native Android build

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>

[grendello]

Tracking issue: #111491

Builds that work (linux and OSX):

• arm64
• x64

Builds that don't (blocked by #111665)

• arm
• x86

Build commands that can be used:

CoreCLR itself

./build.sh -Subset clr.runtime+clr.alljits+clr.corelib+clr.nativecorelib+clr.tools+clr.packages+libs -c Debug|Release -os android -arch <arm64|arm|x64|x86>

No subset (CoreCLR+Mono)

./build.sh -arch <arm64|arm|x64|x86> -os android -c Debug|Release

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[steveisok]

ok, I think this is ready for another review.

@kotlarmilos can you do another official build pass to make sure we don't break / regress?

[MichalStrehovsky]

What is the expected way to invoke build.sh for android? Do we need to pass -cross? If we do need to pass -cross this property should already be set by that.

And if we don't pass -cross: does the code path added in eng/build.sh in this PR kick in? (It seems to be conditioned on -cross being passed to build.sh.)

(So far we have not been passing -cross when building for Android, there is a discussion about that at #56622).

[grendello]

For me, locally, this change breaks the build (and so does passing -cross):

$ ./build.sh clr+libs --os android --arch arm64 -c release --restore --build
...
Commencing CoreCLR Repo build
Error: rootfsDir has been passed, but the location is not valid.

@steveisok are you absolutely sure it's necessary to enable cross build? Using the NDK cmake toolchain definition automatically makes the build a cross one, using the sysroot from the NDK and the appropriate target API+platform libraries and headers.

[janvorli]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.
But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

[MichalStrehovsky]

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though.

This is what #56622 discusses - the existing Android builds (Mono Android/Bionic, native AOT Bionic) do not pass -cross. The build infrastructure doesn't think it's a cross build (it's not considered as a cross build in libs build or in runtime build). It gets away with it because including the Android toolchain definition will make things happen.

The issue discusses switching Android build to the -cross plan but that didn't happen. I don't know if we even have Android build machines that would have the correct ROOTFS_DIR set up. We never built Android as -cross except for the community CoreCLR Android port attempt long time ago (none of the things that actually shipped and created official build specified -cross).

[grendello]

it's necessary to enable cross build

In coreclr, we consider crossbuild to be anything that's not targeting the current architecture / OS for unix based systems. So this should be cross build too. I think there are subtle parts of the build scripts that I think could misbehave if it is not done that way. I am not 100% sure though. But, I am not a fan of feeding in the -cross option automatically. That would make the build for Android experience different from other targets where we always specify the -cross on the command line explicitly.

@grendello looking at your command line, have you set the ROOTFS_DIR env variable too? That's what's necessary for cross builds in general.

No, I have not - to what should I point it? The NDK's toolchains/llvm/prebuilt/${OS}/sysroot directory?

[steveisok]

we've historically never built Android/iOS/Browser with -cross since they all bring a toolchain with custom cmake and rootfs handling, we've only used -cross for "linux" builds. I'm not sure how hard it will be to untangle that.

That's what I mean by not being convinced we need to explicitly set ROOTFS_DIR in the coreclr build. Only passing the toolchain as a cmake argument will do.

[MichalStrehovsky]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

I'm asking because -cross does a lot more than that and passing both -cross and setting up Android cmake toolchain file will cause many things to be set twice. It feels like it will cause maintenance pain.

We could make it so proper crossgen gets picked for Android even without -cross and clean that up when/if the build gets switched to the real -cross plan.

[steveisok]

Is passing -cross needed only to select a different crossgen binary or do we need it for something else?

As far as I can tell, we only need it for crossgen. On osx (and presumably windows), the crossgen/ilcompiler builds will fail because it'll try to link against libjitinterface_arm64.so instead of libjitinterface_arm64.dylib.

[steveisok]

I validated we don't need -cross when building from linux hosts. I pushed a commit to only enable that when on osx for now so that we can move the PR forward.

[steveisok]

Looks like _BuildAnyCrossArch is the property we need with a HostOS != TargetOS condition addition.

[ivanpovazan]

Please add in the description of the PR which host->target builds have been successfully verified, and what build command was used

[ivanpovazan]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

Thanks everyone for your contributions!

[steveisok]

We should exclude official builds of coreCLR on Android for arm and x86 as they are still failing (referenced in the description of the PR and in the #111491) and I think this can be merged.

They are already excluded by this:

<!-- Android 32-bit builds blocked by https://github.com/dotnet/runtime/issues/111665 -->
<_CoreCLRSupportedOS Condition="'$(TargetsAndroid)' == 'true' and '$(TargetArchitecture)' != 'arm' and '$(TargetArchitecture)' != 'x86'">true</_CoreCLRSupportedOS>