Use a list for tests in src/tests/run.py

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.

[jakobbotsch]

Fix#78462

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Fix #78462

Author:	jakobbotsch
Assignees:	jakobbotsch
Labels:	area-Infrastructure-coreclr
Milestone:	-

[jakobbotsch]

Not sure that this is the best fix. It works for other tests already without this change because they disambiguate via the name attribute which HardwareIntrinsics tests don't (see #78462 (comment)). Maybe they should, but given that we already have the assembly name that has the _r/_ro suffix, I assume it's probably not necessary to do.

My main concern is then whether we see the _r/_ro distinction anywhere in AzDO, but I think so, since the assembly name does include that.

[BrianBohe]

Can we change the test names and leave this as a check for future problems?

[jakobbotsch]

Can we change the test names and leave this as a check for future problems?

It's like I said above – I don't think it matters.

[jakobbotsch]

For example, here's one of these test failures:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=81571&view=ms.vss-test-web.build-test-results-tab&runId=1704060&resultId=110825&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab

You can see both HardwareIntrinsics_r as the work item name and when you look at the console log:

BEGINEXECUTION/datadisks/disk1/work/B5A409B3/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=falseHardwareIntrinsics_r.dll ''23:16:14.616Runningtest: global::JIT.HardwareIntrinsics.Arm._AdvSimd.Arm64.Program.Abs_Vector128_Double()
SupportedISAs:

Anyway, I will leave it up to @trylek and @davidwrighton whether they think this fix is ok, or if we should try to disambiguate the hardware intrinsics test names by assembly also. I'm not sure where to do the latter after #74886.

[BruceForstall]

LGTM, but feel free to wait for @trylek and/or @davidwrighton for more context

[trylek]

Nice, thanks for fixing this!

[davidwrighton]

I think this change is fine, but I'll admit I with the testresults.xml file had the distinction in it itself.