Added 'standalone' option when building tests for ilasm roundtrip by TIHan · Pull Request #93037 · dotnet/runtime

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

TIHan · 2023-10-04T23:14:55Z

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

ghost · 2023-10-04T23:15:11Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	`area-Infrastructure-coreclr`
Milestone:	-

markples · 2023-10-05T17:13:02Z

Can you please show a lab run with this ilasm flag running?

markples · 2023-10-05T17:15:15Z

- - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS
- displayName: Build managed test components
+ - ${{ if in(parameters.testGroup, 'ilasm') }}:
+ - script: $(Build.SourcesDirectory)/src/tests/build$(scriptExt) $(logRootNameArg)Managed allTargets skipnative skipgeneratelayout standalone skiptestwrappers $(buildConfig) $(archType) $(runtimeFlavorArgs) $(crossArg) $(priorityArg) $(testTreeFilterArg) ci /p:TargetOS=AnyOS


I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

markples · 2023-10-05T17:21:17Z

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

TIHan · 2023-10-05T20:29:37Z

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

markples · 2023-10-05T20:42:28Z

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

TIHan · 2023-10-05T22:51:40Z

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

…ld-as-standalone

markples · 2023-10-06T04:45:51Z

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

TIHan · 2023-10-06T17:43:05Z

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

markples · 2023-10-06T18:00:11Z

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

TIHan · 2023-10-06T18:18:13Z

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

markples · 2023-10-06T18:50:34Z

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd
Return code: 0
Raw output file: C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

TIHan · 2023-10-06T20:24:40Z

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

TIHan · 2023-10-06T22:54:01Z

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

markples · 2023-10-06T23:16:45Z

2 things going on here

Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

…ld-as-standalone

TIHan · 2023-10-10T00:29:27Z

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

TIHan · 2023-10-10T00:32:36Z

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan · 2023-10-11T21:54:21Z

Closing in favor of #93368

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Added 'standalone' option when building tests
21cf299

ghost assigned TIHan Oct 4, 2023

ghost added the area-Infrastructure-coreclr Only use for closed issues label Oct 4, 2023

Fix sh env var
e3ff1fd

markples reviewed Oct 5, 2023
View reviewed changes

build-analysisBot mentioned this pull request Oct 5, 2023
LibraryImportGenerator unit test segfault illegal memory access #89054
Closed

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
f9d86c4
…ld-as-standalone

build-analysisBot mentioned this pull request Oct 6, 2023
Build crashes in System.Runtime.Serialization.SerializationGuard #92007
Closed

TIHan added 2 commits October 9, 2023 13:41

Merge remote-tracking branch 'upstream/main' into ilasm-roundtrip-bui…
b3a2d2a
…ld-as-standalone

Fix provider test
9b3fa8f

build-analysisBot mentioned this pull request Oct 10, 2023
Microsoft.Gen.OptionsValidation.Test.EmitterTests.TestEmitter fails occasionally #93229
Closed

TIHan added 3 commits October 9, 2023 17:40

Update build.sh
7fb4c9e

Revert provider
2db8fb6

Merge branch 'ilasm-roundtrip-build-as-standalone' ofhttps://github.…
41ee8d3
…com/TIHan/runtime into ilasm-roundtrip-build-as-standalone

TIHan closed this Oct 11, 2023

ghost locked as resolved and limited conversation to collaborators Nov 11, 2023

Uh oh!

Conversation

TIHan commented Oct 4, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Oct 4, 2023

Uh oh!

markples commented Oct 5, 2023

Uh oh!

markplesOct 5, 2023

Choose a reason for hiding this comment

Uh oh!

markples commented Oct 5, 2023

Uh oh!

TIHan commented Oct 5, 2023

Uh oh!

markples commented Oct 5, 2023

Uh oh!

TIHan commented Oct 5, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 10, 2023

Uh oh!

TIHan commented Oct 10, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TIHan commented Oct 11, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

TIHan commented Oct 4, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Oct 4, 2023

Uh oh!

markples commented Oct 5, 2023

Uh oh!

markplesOct 5, 2023

Choose a reason for hiding this comment

Uh oh!

markples commented Oct 5, 2023

Uh oh!

TIHan commented Oct 5, 2023

Uh oh!

markples commented Oct 5, 2023

Uh oh!

TIHan commented Oct 5, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

markples commented Oct 6, 2023

Uh oh!

TIHan commented Oct 6, 2023

Uh oh!

markples commented Oct 6, 2023

TIHan commented Oct 4, 2023•
edited
Loading

TIHan commented Oct 10, 2023•
edited
Loading

TIHan commented Oct 4, 2023•
edited
Loading

TIHan commented Oct 10, 2023•
edited
Loading

TIHan commented Oct 4, 2023•
edited
Loading