Fix build on Clear Linux and Android device

[am11]

Consists of two small commits:

1. Fix native component build for Clear Linux

   • Distros such as Clear Linux have some compiler flags set in the environment in the default, factory setting. One of the flag is _FORTIFY_SOURCE=2 which conflictst with CoreCLR and Libraries compilation and break the build. this delta clears _FORTIFY_SOURCE=2 from CMAKE_CXX_FLAGS and CMAKE_C_FLAGS if set by the environment (i.e. very early in the process).
   • FixesBuild errors under libraries/native when building with GCC #2188

2. Only skip call to check_pie_supported on Android

   • NDK with API level 21 is required by Xamarin, NDK with API level 28 is required by CoreCLR for glob(3), see the comment:

     runtime/eng/common/cross/build-android-rootfs.sh

     Line 23 in f79afdb

     __ApiLevel=28 # The minimum platform for arm64 is API level 21 but the minimum version that support glob(3) is 28. See $ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/glob.h

     with level 21, linker (ld) from LLVM toolchain does not recognize -no-pie flag, thus the call to check_pie_supported fails. However, it sets the PIC flag for us (not sure exactly where?). With level 28, it allows -no-pie, and also sets PIC flag. Hence, the current master is happy for both use-cases.
   • When building on physical device or emulator, although Clang 9.0.1 acquired from termux packages supports -no-pie, it does not set PIC (as in that case we do not take NDK provided android.toolchain.cmake into use).
   • To make all three use-cases (Xamrian, rootfs and device) work, we only need to skip the call to check_pie_supported, and set PIC unconditionally (as Android requires PIC and setting it twice in certain scenario has no side-effect).

cc @jkotas, @akoeplinger, @janvorli

[akoeplinger]

The Android change looks good but I'm not so sure about the CFLAGS/CXXFLAGS one.

The reason why Clear Linux and other distros set these CFLAGS is exactly because they want them to be used by projects.

It is standard behavior of autotools-based projects to be able to inject your own CFLAGS via env variables.

[am11]

Thanks. This project has complex configuration graph and desired flags are anyway materialized in configuration files, so some/most of the C{XX}Flags from environment are overwritten (unless cmake cache variable is used?)

On Clear Linux, main problem is that _FORTIFY_SOURCE level is set to 2, and we are not (yet) _FORTIFY_SOURCE complaint.

If support of C{XX}Flags from the environment is desired (which I agree is the conventional approach), we can:

1. just unset _FORTIFY_SOURCE (short-term)
2. make sources _FORTIFY_SOURCE complaint (long-term)

or do nothing. 🙂

[akoeplinger]

If unsetting _FORTIFY_SOURCE fixes the compilation issue then that sounds like the best approach until it can be made fortify compliant.

[am11]

New changes pushed:

• CoreCLR and installer builds were happy with GCC 9 on Clear Linux, upon clearing the _FORTIFY_SOURCE=2 from CFLAGS and CXXFLAGS.
  • -skiprestoreoptdata is a no-op and misleading entry in help menu: build-runtime.sh -?. The correct option is -nopgooptimize which offsets what skiprestoreoptdata was supposed to provide and then some. So I deleted this line (as I realized the skip option does not work, when tried using it for fast compilation, and switched to -nopgooptimize).
  • -Wno-format-security is a required suppression when we suppress -Wformat for libunwind, according to CFLAGS set in Clear Linux.
• libraries required a little code changes pertaining to formatting. I have avoided adding suppression on project level in one place for incompatible enums, as we are using in-place (#pragma-based) suppression for clang in multiple places.

[am11]

Seems like a disconnection issue on CI:

##[error]We stopped hearing from agent Azure Pipelines 24. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610

@jkotas, could you please run the failing legs?

[am11]

@akoeplinger, updated the top comment. Could you please give it another pass? :)

[am11]

@jkotas, @janvorli, could you please take a look?

[janvorli]

LGTM, thank you!

[janvorli]

LGTM, thank you!

[am11]

Although very close to the area which this PR is touching, but Libraries Build iOS arm Debug error seems unrelated

 Invoking "/Users/runner/runners/2.165.2/work/1/s/eng/native/gen-buildsys.sh" "/Users/runner/runners/2.165.2/work/1/s/src/libraries/Native/Unix" "/Users/runner/runners/2.165.2/work/1/s/src/libraries/Native/Unix" "/Users/runner/runners/2.165.2/work/1/s/artifacts/obj/native/netcoreapp5.0-iOS-Release-arm" arm clang "" "" Release "" -DCLR_ENG_NATIVE_DIR="/Users/runner/runners/2.165.2/work/1/s/eng/native" -DCMAKE_OSX_SYSROOT=iphoneos -DCMAKE_OSX_DEPLOYMENT_TARGET=7.0 -DCMAKE_OSX_ARCHITECTURES="armv7;armv7s" -DCMAKE_SYSTEM_NAME=iOS CMake Warning at /usr/local/Cellar/cmake/3.17.0_1/share/cmake/Modules/Platform/Darwin-Initialize.cmake:204 (message):
No SDK found for architecture 'armv7'
Call Stack (most recent call first):

[jkotas]

OSX machine recycled during the run (#34472)

[jkotas]

@akoeplinger Are you ok with this change after the discussion?

[am11]

@jkotas, can this be merged?

[jkotas]

I am waiting for @akoeplinger to clear his changes requested feedback. The merging is blocked because of it.

It is possible to merge anyway using admin privileges, but I prefer to use my admin privileges to handle emergencies only.

[am11]

Linux musl arm64 release looks similar to dotnet/arcade#4190. Could be due to #34044 (https://pkgs.dev.azure.com/dnceng/internal/ change).

cc @mmitche: https://dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_apis/build/builds/592074/logs/1016

 Response status code does not indicate success: 429 (Request was blocked due to exceeding usage of resource 'ATCPU' in namespace 'IPAddress'. For more information on why your request was blocked, see the topic "Rate limits" on the Microsoft Web site (https://go.microsoft.com/fwlink/?LinkId=823950). (DevOps Activity ID: 0CB2927A-F90D-4A18-B207-8C7F3F0C9D3B)).

[akoeplinger]

Looks good to me apart from one question. Sorry for the delay.

[akoeplinger]

@am11 this seems like an unintentional change?

[am11]

@akoeplinger, I called it out here #34211 (comment). This option in help menu was removed from code, the case-esac block which handles options, and superseded by -nopgooptimize. For example, to quickly build coreclr native components with PAL tests on an unsupported Unix (e.g. SunOS, QNX), where dontet(1)/msbuild is currently unavailable, we can use:

./src/coreclr/build-runtime.sh -skipgenerateversion -nopgooptimize \
-cmakeargs -DCLR_CMAKE_BUILD_TESTS=1

[akoeplinger]

Ok. It's still in src/coreclr/build-runtime.cmd that's why I was confused :)

[am11]

Remaining failures are unrelated to PR changes.

Installer Build and Test Linux_musl_x64 Debug failure is #34590 and #33675.

Libraries Test Run release mono Linux x64 Debug interupped while downloading/extracting the test result archive from Helix:

Extraction failed for file: /datadisks/disk1/workspace/_work/1/s/__download__/libraries_test_assets_Linux_x64_Debug/libraries_test_assets_Linux_x64_Debug.tar.gz code: 2 stdout: helix/
helix/tests/
helix/tests/Linux.AnyCPU.Debug/
helix/tests/Linux.AnyCPU.Debug/System.ComponentModel.Composition.Tests.zip
...
helix/tests/Linux.AnyCPU.Debug/System.IO.Compression.Brotli.Tests.zip
stderr: gzip: stdin: unexpected end of file
/bin/tar: Unexpected EOF in archive
/bin/tar: Unexpected EOF in archive
/bin/tar: Error is not recoverable: exiting now
error: undefined;