Shift Most of Wasm AOT test build to helix

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.

[steveisok]

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

This is done by:

1. building the test assemblies on the build machine
   • the wasm part of the build is not executed on the build machine,
     because it has the AOT build part
2. Zip up the test assembly+friends, and any bits required to run the wasm
   app build for that on helix (eg. emsdk, wasm app targets, cross compiler etc)
3. Send all this to helix, and use a custom aot-build.proj
   • which recreates all the build inputs for the WasmBuildApp target
     using the paths for the assets on helix
   • then we can run WasmBuildApp for the build, resulting in a wasm app
     bundle.
4. Run the tests!

• We already have the bits required for building wasm apps on helix, supported
  for Wasm.Build.Tests, which we can use here too.

Trimming:

• Since, AOT can be so expensive, we use EnableAggressiveTrimming=true(EAT), but
  that means that we could have issues due to trimming.

• And it can sometimes be unclear whether the build/test failures are due to trimming
  or AOT.

• Because these builds+test runs are different from other builds, owing to the
  "build partially on helix" step, a normal EAT build would not be the same as

• to help with testing this, we add two lanes to runtime-staging:

  • *_Mono_AOT: builds AOT+EAT on helix
  • *_Mono_EAT: builds EAT, on helix
    • this is required because we want to run almost the same kinda
      build: 1. build test assembly; 2. send to helix; 3. build wasm app; 4. run tests

• This should effectively mean that we can see which errors might be due to EAT, and
  which are clearly because of EAT+AOT.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since AOT'ing each test suite takes between 3-9 min, we need to shift the burden over to helix.

Author:	steveisok
Assignees:	-
Labels:	area-Infrastructure-mono
Milestone:	-

[radical]

Some these need to have values different from what UseDefaultBlazorWasmFeatureSwitches wants:

• DebuggerSupport - this is needed by some library tests because they depend on pdbs, or debugger attributes
  • Currently, I am setting this in the specific projects that need it
• UseSystemResourceKeys - IIUC, this is needed so that the resources don't get trimmed, which is needed by some library tests

Setting these only for some library projects makes it kinda inconsistent. And I'm wondering if that affects what the intention of UseDefaultBlazorWasmFeatureSwitches is.
So, do we need switch at all? If so, then what should be done here?

/cc @lewing@akoeplinger

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[radical]

@steveisok@mdh1418 Do you know why this change was made? main is still using the older version.

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[radical]

Failing test in runtime (Libraries Test Run release mono Linux x64 Debug): #51588

Failing test in Libraries Test Run release coreclr Linux x64 Debug is possibly: #48658

Failing test in Libraries Test Run release mono OSX x64 Debug: #51611

[akoeplinger]

judging from the date it's to bring in dotnet/dotnet-buildtools-prereqs-docker@f25f6cd

[akoeplinger]

I agree with @radical, the intention of UseDefaultBlazorWASMFeatureSwitches is to make us consistent with Blazor. I think overriding in the individual library test projects where necessary is fine for now.

[akoeplinger]

do we have a way of knowing the list of properties is complete or when we need to update it?

[radical]

Um .. not really. So, when something breaks? I didn't add all the properties either, because I want to do that as it comes up.

[akoeplinger]

curious why does this need .pdb?

[radical]

Eg:

 fail: [FAIL] System.Collections.Tests.HashtableTests.DebuggerAttribute
info: System.InvalidOperationException : Expected one DebuggerDisplayAttribute on System.Collections.Hashtable.
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerDisplayReferences(Object obj)
info: at System.Collections.Tests.HashtableTests.DebuggerAttribute()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
fail: [FAIL] System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException
info: System.InvalidOperationException : Expected one DebuggerTypeProxyAttribute on System.Collections.Queue.
info: at System.Diagnostics.DebuggerAttributes.GetProxyType(Type type, Type[] genericTypeArguments)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Type[] genericTypeArguments, Object obj)
info: at System.Diagnostics.DebuggerAttributes.ValidateDebuggerTypeProxyProperties(Type type, Object obj)
info: at System.Collections.Tests.QueueTests.DebuggerAttribute_NullQueue_ThrowsArgumentNullException()
info: at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

[akoeplinger]

this should ideally be added to

runtime/src/tasks/tasks.proj

Lines 8 to 11 in d08f7f2

	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmAppBuilder\WasmAppBuilder.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />
	<ProjectReferenceRemove="$(MSBuildThisFileDirectory)WasmBuildTasks\WasmBuildTasks.csproj"
	Condition="'$(TargetOS)' != 'Browser'" />

[radical]

I'm not sure I understand. This file will get built with WasmBuildTasks. What should be added to tasks.proj?

[lewing]

looks green to me, there is some minor whitespace damage but this was a long road so lets land it.