Make Ninja the default CMake generator on Windows for the repo

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.

[jkoritzinsky]

Part of April's batched infrastructure rollout: #49889

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes#45955
Contributes to #44761
Contributes to #47022

If a developer wants to easily create an MSBuild solution for CoreCLR's native code and open it in VS, they can run ./build.cmd -vs coreclr.sln -c <config> -a <arch> to create and open a solution for the given arch and config in VS.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR switches Ninja to be the default generator on Windows for all native build components.

MSBuild is still available on Windows with the -msbuild flag. To prevent conflicts from using both the Ninja and MSBuild generators in the same dev workflow, I've moved the MSBuild-generated sources to an ide subfolder of each native build's intermediate directory.

Fixes #45955
Contributes to #44761

Since this fixes #45955, it might require a jitutils update as well. cc @dotnet/jit-contrib

This requires an infrastructure rollout to merge as it is an infra behavior change.

Author:	jkoritzinsky
Assignees:	-
Labels:	area-Infrastructure
Milestone:	6.0.0

[safern]

Looks reasonable.

[MichalStrehovsky]

Out of curiosity, why are we making this the default before CMake in VS user story is done (I see it's doesn't have the checkmark in #44761). Is it that much faster?

A lot of people (me included) use the generated projects to do work, but I don't use them on a daily basis. I can see myself hitting this a couple times until I reach the "acceptance" stage of grief:

1. build.cmd blah blah
2. Work on something that is not necessarily "the runtime" (crossgen2, NativeAOT, CoreLib work, etc.)
3. Figure out "oh, I need to open RyuJIT/the runtime in VS" because I don't like reverse engineering 3 layers of macros with findstr.
4. Oh shoot, I don't have the solution files, or they're out of sync because I forgot to pass -msbuild
5. Rebuild everything with vcxproj. Get coffee. Complain about build times to the toaster in the kitchen.

I invoke build.cmd manually a lot. Adding another mandatory parameter is a bit of a nuisance. It feels like the defaults in build.cmd are optimized for machines (the CI), not for humans. I would accept this if ninja is like 50% faster, but I don't know how much faster it is.

[jashook]

Lgtm. Are the binaries produced identical? If not what is the size difference?

[hoyosjs]

Couple small things. Also, this might be really rough on dev inner loop, if that many people use the SLN + proj files generated. Agree on the rollout part.

[jkoritzinsky]

@MichalStrehovsky in addition to providing a speedup, Ninja can further enable #47022 by allowing us to let Ninja and CMake do an up to date check on the CMake build files instead of reconfiguring every build (and relying on CMake to not write out build files that didn't change).

For your use case, there's a (apparently little known) feature in the root build scripts:

If you run ./build -vs coreclr.sln, it will generate and open the VS solution for a full CoreCLR build. This also respects the flags for the configuration and architecture that the build script uses. With this switch, you never even need to go searching in the obj directories to find the solution file manually.

With the -vs coreclr.sln support, I don't worry too much about the lack of completion of the CMake-in-VS experience for the repo. Does the -vs coreclr.sln support satisfy your use case?

[jkoritzinsky]

@jashook The last set of comparison numbers I had (#41897 (comment)) showed Ninja sizes as comparable if not better.

[MichalStrehovsky]

Does the -vs coreclr.sln support satisfy your use case?

Yup, looks like it will do the trick. Thank you!

[BruceForstall]

I notice there are a ton of compiler warnings in the build. Can these get addressed before merge?

e.g.,

https://dev.azure.com/dnceng/public/_build/results?buildId=1044403&view=logs&j=388cb898-bfbc-5d7d-00fc-4978afab125f&t=e474b4ca-e4ef-54e1-ea7b-d237313199c0

Lots of:

cl : Command line warning D9025 : overriding '/W3' with '/W4'

and:

cl : Command line warning D9025 : overriding '/EHa' with '/EHs'

and:

cl : Command line warning D9025 : overriding '/Zc:wchar_t' with '/Zc:wchar_t-'

[jkotas]

Also, can we enable warnings as errors? I believe that the msbuild has warnings as errors enabled right now.

[BruceForstall]

Does ninja generate any build log files? I can't find any.

If I have a compiler error in the JIT, the build-runtime.cmd script outputs:

BUILD: Error: native component build failed. Refer to the build log files for details.
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.log"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.wrn"
"C:\gh\runtime3\src\coreclr\..\..\artifacts\log\Debug\CoreCLR_windows__x64__Debug.err"

but those files aren't there.

(Note, these are experiences without picking up this PR, which I have been assuming aren't fixed by this PR)

[jkoritzinsky]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

I'll work on driving the warnings to 0 before merging this in.

[jkoritzinsky]

Warnings-as-errors is also enabled in the Ninja build, but this class of warnings are apparently excluded from warnings-as-errors behavior by CL

[BruceForstall]

I don't believe ninja generates any build log files by default. Similar to make, it stops on the first error so it's pretty easy to find the errors that caused the build failures.

If there are no log files, how do you debug issues with the build itself? ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

[jkoritzinsky]

The generated build.ninja file has all of the flags, dependencies, and command lines that ninja uses to build every file and target in the build. The file format is very simple and straightforward (very similar to a simple Makefile), so it is pretty easy to figure out where to look to check flags.

[jkotas]

warnings

The release build has still a bunch of warnings in the last CI run like:

2021-03-19T19:58:13.4594871Z cl : Command line warning D9025 : overriding '/Ox' with '/O1'
2021-03-19T19:58:13.4596777Z cl : Command line warning D9025 : overriding '/O2' with '/O1'

[jkoritzinsky]

I must have missed those last time I looked. I’ll do another pass over the warnings now that everything builds successfully.

[akoeplinger]

ninja doesn't output the full list of commands and options to the command-line (nor would you want it to). What if I want to see exactly what the compile options are (e.g., warning levels, optimization levels, etc.)?

@BruceForstall if something fails you should see the full compiler command line including all options:

 [226/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-pinning.c.o
[227/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-nursery-allocator.c.o
[228/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/sgen/sgen-memory-governor.c.o
[229/346] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
FAILED: mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o
/Library/Developer/CommandLineTools/usr/bin/clang -DHAVE_CONFIG_H -DMONO_DLL_EXPORT -D_THREAD_SAFE -DOSX_ICU_LIBRARY_PATH=\"/usr/lib/libicucore.dylib\" -Imono/mini -Imono/mini/../.. -Imono/mini/../../mono/eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/.. -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eglib -I/Users/alexander/dev/runtime/src/mono/mono/mini/../sgen -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../native -I/Users/alexander/dev/runtime/src/mono/mono/mini/../eventpipe -arch arm64 -std=gnu99 -fno-strict-aliasing -fwrapv -Wall -Wunused -Wmissing-declarations -Wpointer-arith -Wno-cast-qual -Wwrite-strings -Wno-switch -Wno-switch-enum -Wno-unused-value -Wno-attributes -Wmissing-prototypes -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wno-unused-function -Qunused-arguments -Wno-tautological-compare -Wno-parentheses-equality -Wno-self-assign -Wno-return-stack-address -Wno-constant-logical-operand -Wno-zero-length-array -Werror=return-type -Werror-implicit-function-declaration -g -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk -mmacosx-version-min=11.0 -g -fPIC -fvisibility=hidden -I -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/System.Globalization.Native/ -I/Users/alexander/dev/runtime/src/mono/mono/mini/../../../libraries/Native/Unix/Common/ -DTARGET_UNIX -DU_DISABLE_RENAMING -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-switch-enum -Wno-covered-switch-default -Wno-extra-semi-stmt -Wno-unknown-warning-option -Wno-deprecated-declarations -MD -MT mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -MF mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o.d -o mono/mini/CMakeFiles/monosgen-objects.dir/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c.o -c /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c:8:
In file included from /Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_locale_internal.h:6:
/Users/alexander/dev/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_icushim_internal.h:22:10: fatal error: 'unicode/ucurr.h' file not found
#include <unicode/ucurr.h>
^~~~~~~~~~~~~~~~~
1 error generated.

If you want to see the options for a specific compilation unit without a failure then you can either look into the generated build.ninja file or pass --verbose option to ninja.