Enable using the Ninja generator instead of Visual Studio on Windows builds. by jkoritzinsky · Pull Request #41897 · dotnet/runtime

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky · 2020-09-04T21:57:53Z

Summary for Infra Rollout

We're enabling users to use Ninja across the coreclr, libraries, and host native builds instead of Visual Studio or Make. Ninja is a slimmed down build orchestrator that performs much faster than MSBuild on our native Windows CoreCLR build and performs comparably to MSBuild on the Windows libraries and host builds and Make on all non-Windows builds.

You can use the -ninja flag to use Ninja to run the native builds instead of MSBuild or Make. For this rollout, we're not changing the default experience for local development, but we are enabling Ninja builds in CI for the CoreCLR build.

There is one breaking change and one non-breaking change for developers:

Breaking: If you are using the src/coreclr/build-runtime.cmd script to build on Windows, you'll need to build either the clr.dactools or clr.runtime subset once from the root script first. The DacTableGen tool used in the Windows build has moved out of the CMake build and is now a regular C# project. As a result, it needs to be built via the root build script once so it is available for the native build.
Non breaking: The minimum recommended CMake version is 3.16. If you use CMake 3.16 precompiled headers will be used in all builds. If you use CMake 3.15.5 or lower, precompiled headers will not be used. This is a behavior change, since we used to always have precompiled headers on Windows and never have them on non-Windows. If you notice slowness in your Windows build after pulling down this PR and have not updated CMake, try installing CMake 3.16 to re-enable precompiled headers.

Original Summary

This PR enables using the Ninja generator on Windows instead of the Visual Studio CMake generators via the -ninja switch (for the build-runtime.cmd script) and via a Ninja MSBuild global property. To keep pre-compiled headers working on Windows, this PR bumps the minimum recommended CMake version to 3.16. Using an older CMake version is still possible, but your build will not support precompiled headers. However, now precompiled headers are enabled on all platforms (not just Windows) when using CMake 3.16 or higher.

Since the C# support in CMake is only for the Visual Studio generators, I've moved the DacTableGen project into a subset that's included with the clr.runtime subset and also moved the PIA for DIALib into a checked-in ilproj project by decompiling it with ildasm.

To support opening the CoreCLR VS-generated solution in Visual Studio, I've augmented the root -vs flag. If you pass coreclr.sln (case-insensitive) to the -vs flag, the build script will generate the CoreCLR solution for the provided parameters using the VS generator (if it hasn't been generated already) and open the solution in VS.

I've updated the CoreCLR, Libraries, and Host builds to all enable using Ninja.

For this PR, Ninja usage will be opt-in. In the future, we will make Ninja the default generator and make the Visual Studio generator opt-in.

TODO:

CI validation
- The CI machines don't have Ninja installed yet, so this is blocked on getting it installed.
Documentation of new options

…blegen-csproj

… instead of via CMake.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…ninja-windows

…-windows

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…flag.

… share windows cmake configure infra.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

…urning on /W3 and disabling any new warnings we hit.

…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky · 2020-10-22T18:39:37Z

This PR is ready for review!

ViktorHofer · 2020-10-22T19:30:21Z

 <Target Name="BuildNativeUnix"
 BeforeTargets="Build"
- Condition="'$(TargetOS)' != 'Windows_NT'">
+ Condition="!$([MSBuild]::IsOsPlatform(Windows))">


Unsure if this ever worked but the TargetOS check doesn't assert the current OS but the target one that is passed in via the -os switch.

Can I build the native bits targeting Unix on Windows?

This broke with the WASM on Windows build, since it was going down the build-native.sh path when it should have gone down the build-native.cmd path.

Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

BruceForstall · 2020-10-28T01:04:19Z

Looks like you need to add more documentation on Ninja, e.g.:

From where do I install it?
How do I install it?
What is the minimum version?

The answers to these questions need to cover all platforms.

The documents at the various "Requirements" links on https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md should be updated.

BruceForstall · 2020-10-28T01:06:35Z

The minimum recommended CMake version is 3.16

I didn't look thoroughly: is there an easily visible warning on the console if you don't have the recommended minimum cmake version? (i.e., it seems like there should be)

jkoritzinsky · 2020-10-28T17:30:29Z

We have an error for the minimum required CMake, but not a warning for the minimum recommended CMake (this is the first time they differ). I'll add a warning output.

…Make is older than 3.16.0.

jkoritzinsky · 2020-10-30T17:57:33Z

Test failures are #43983 and #43927

jkoritzinsky added 27 commits January 8, 2020 16:44

Move DacTableGen out of the CMake build and into tools-local.
cad3f29

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto dacta…
e921ecc
…blegen-csproj

Attempt: usenmakefiles means ninja.
a69c346

Make changes to Windows build to support non-VS generators like ninja.
d1b12b5

Use CMake 3.16 PCHs.
df589ef

Remove explicit process count.
0524739

Build and run DacTableGen via the .NET CLI we pull down for the build…
e99246f
… instead of via CMake.

Update configurecompiler.cmake to specify language (needed for Ninja).
03140d8
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Merge branch 'ninja-windows' of github.com:jkoritzinsky/runtime into …
9bdf0f1
…ninja-windows

Get Ninja build working again.
417b07a

Merge branch 'master' ofhttps://github.com/dotnet/runtimeinto ninja…
db7454a
…-windows

Pass configuration to cmake configure step for Ninja build.
b70eeca

Rename flag
36cc91e

Fix configure step for cross-arch native build
fe484c9
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Support building via Ninja for coreclr via an MSBuild property
cba81df
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Implement support for opening the CoreCLR solution in VS via the -vs …
c20b356
…flag.

Fix setting generator to Ninja.
86f5f90

Merge libraries gen-buildsys-win.bat with runtime gen-buildsys.cmd to…
a2478cc
… share windows cmake configure infra.

Enable building libraries native build with ninja.
ca84dd2

Centralize setting CMAKE_SYSTEM_VERSION for windows builds.
3b17211
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Update host build to use shared gen-buildsys.cmd
8422b09

Fix multi-config generator detection.
885974e
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Exit with success when we build successfully.
a8feb74
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Enable and fix up Ninja build for hosts.
e7b9280
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Only link delayimp.lib on Windows
6bcaec4
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Remove "overriding /W3 with /W1" warning in Ninja build of hosts by t…
cb1f6a1
…urning on /W3 and disabling any new warnings we hit.

Reduce command line override warnings by adopting the MSVC_RUNTIME_LI…
03c4cc9
…BRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

jkoritzinsky added the area-Infrastructure label Sep 4, 2020

jkoritzinsky added this to the 6.0.0 milestone Sep 4, 2020

jkoritzinsky requested a review from a teamSeptember 4, 2020 21:57

Fix 2-phase cross targeting to do one phase at a time and initialize …
cfc26f5
…the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

jkoritzinsky changed the title ~~[WIP] Enable using the Ninja generator instead of Visual Studio on Windows builds.~~Enable using the Ninja generator instead of Visual Studio on Windows builds.Oct 22, 2020

jkoritzinsky marked this pull request as ready for review October 22, 2020 18:39

Restore old /Ox optimization setting for Release.
343690e

ViktorHofer reviewed Oct 22, 2020
View reviewed changes

jkoritzinsky mentioned this pull request Oct 22, 2020
Infrastructure - Rollout (November 2020) #43351
Closed

janvorli reviewed Oct 22, 2020
View reviewed changes

jkoritzinsky added 4 commits October 22, 2020 16:46

Fix typos, don't pass exception flags to C files.
9af32b3

Remove unneeded pragma.
61e4c81

Merge branch 'master' of github.com:dotnet/runtime into ninja-windows
a437115

Fix .pgd file installation.
83939c6
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

janvorli approved these changes Oct 27, 2020
View reviewed changes

Add mention of the -ninja flag in the Building CoreCLR documentation.
f95c8f6
Signed-off-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Mention installation locations for Ninja and warn while building if C…
ff80bab
…Make is older than 3.16.0.

BruceForstall reviewed Oct 28, 2020
View reviewed changes

Comment threaddocs/workflow/requirements/windows-requirements.md Outdated

Update docs/workflow/requirements/windows-requirements.md
4db3105

This was referenced Oct 28, 2020
OSX deprovision jaredpar/runfo#41
Closed
OSX machines are de-provisioned during CI / PR runs leading to failures #34472
Closed

jkoritzinsky merged commit 8c6a049 into dotnet:masterNov 2, 2020

jkoritzinsky deleted the ninja-windows branch November 2, 2020 15:50

stephentoub mentioned this pull request Nov 2, 2020
Unable to build dotnet/runtime due to cmake error about MultiThreadedDebug #44152
Closed

hoyosjs mentioned this pull request Nov 4, 2020
CoreCLR missing resources needed for managed debugging on Windows #44232
Closed

AndyAyersMS mentioned this pull request Nov 11, 2020
[Perf -14%] Microsoft.Extensions.Logging.LoggingOverhead (3) DrewScoggins/performance-2#3249
Open

AndyAyersMS mentioned this pull request Nov 19, 2020
Consume new PGO / IBC data from optimization repo #44925
Closed

ghost locked as resolved and limited conversation to collaborators Dec 7, 2020