CI matrix change: Windows by aik-jahoda · Pull Request #58989 · dotnet/runtime

aik-jahoda · 2021-09-11T17:14:36Z

Changes in CI matrix:

OS	What to do	Notes
Windows 7 (x86)	Drop PR (rolling sufficient)
Windows 8.1	Drop PR in favor of W10+
Windows 10 19H1 (server as proxy for client) (x86 and 64)	As far as possible, remove x64 rolling in favor of W11.
Windows Server RS5 (1809)	Drop it (EOL)

part of #57947

ghost · 2021-09-11T17:14:42Z

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

wfurt

generally look ok to me. It is nice to see Windows7 go.

…windows

safern · 2021-09-21T14:54:07Z

 - ${{ if notIn(parameters.jobParameters.framework, 'net48') }}:
 - ${{ if and(eq(parameters.jobParameters.testScope, 'outerloop'), eq(parameters.jobParameters.runtimeFlavor, 'mono')) }}:
- - Windows.81.Amd64.Open
 - Windows.10.Amd64.Server19H1.Open


Do we want to keep this one in Rolling as well to measure pass rate of the pipeline and match PR with Rolling as much as possible?

/cc: @ViktorHofer

The original idea was to mix PR and rolling to save resources but with @jkotas's proposal, we will run rolling builds less often and should probably always the full matrix (of OS + arch + configuration) on rolling builds.

…windows

… jajahoda/libmatrix-windows

aik-jahoda · 2021-09-29T13:14:05Z

@safern, @jkoritzinsky , are you ok with this matrix change?

safern · 2021-09-30T23:07:04Z

 - Windows.81.Amd64.Open
- - Windows.10.Amd64.ServerRS5.Open
- - Windows.10.Amd64.Server19H1.Open
+ - Windows.10.Amd64.Server19H1.ES.Open


Could you leave the .ES. one on both PRs and rolling? We get english coverage on Windows.11.
FYI @danmoseley

Currently Windows.10.Amd64.Server19H1.ES.Open is on both PR and Rolling. Or did I mess the isFullMatrix conditions?

Another option is to mix in a different locale. @AlexRadch has made several fixes to tests so they pass in ru-RU. Another contributor has fixed issues in ko-KR. Clearly ES isn't covering it all. So we could have JP or KO.
Up to you guys.

For that we would need a new helix queue. So we can get that request up to change the .ES. machines to another locale, I like that idea.

Currently Windows.10.Amd64.Server19H1.ES.Open is on both PR and Rolling. Or did I mess the isFullMatrix conditions?
You are right. I miss read the change.

…windows

ghost · 2021-11-01T08:11:00Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

Changes in CI matrix:

OS	What to do	Notes
Windows 7 (x86)	Drop PR (rolling sufficient)
Windows 8.1	Drop PR in favor of W10+
Windows 10 19H1 (server as proxy for client) (x86 and 64)	As far as possible, remove x64 rolling in favor of W11.
Windows Server RS5 (1809)	Drop it (EOL)

part of #57947

Author:	aik-jahoda
Assignees:	-
Labels:	`area-Infrastructure`
Milestone:	-

…windows

karelz · 2021-11-29T15:53:04Z

Update: Chasing down one last timeout problem. Using testing PR #61970 with additional legs.

aik-jahoda · 2021-12-08T17:03:13Z

@GrabYourPitchforks, do you have an idea why is this test failing on Windows 11 (x86 library test)? Are there any OS prerequisites for the test?
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-jajahoda-libmatrix-5e681f823efa4a3e8b/System.Text.Encodings.Web.Tests/1/console.95c30e0d.log?sv=2019-07-07&se=2021-12-26T15%3A34%3A41Z&sr=c&sp=rl&sig=OL7%2FjCWh8CqpIwnNF0vcXSssUrDYhx85alvT4QDHCqg%3D

aik-jahoda · 2021-12-14T14:53:08Z

cc @eiriktsarpalis @layomia

Do you have any idea why the ForbidUndefinedCharacters_RemovesUndefinedChars fails? Are there any OS prerequisites for the test?

Console log: 'System.Text.Encodings.Web.Tests' from job 5e681f82-3efa-4a3e-8b11-6b0a8f255225 workitem eb33ae1d-02cf-4f87-ada1-9eb34701aab9 (windows.11.amd64.clientpre.open) executed on machine a002YFC
C:\h\w\AFB00942\w\BE220A24\e>taskkill.exe /f /im corerun.exe ERROR: The process "corerun.exe" not found.
C:\h\w\AFB00942\w\BE220A24\e>call RunTests.cmd --runtime-path C:\h\w\AFB00942\p ----- start Mon 12/06/2021 15:46:18.12 =============== To repro directly: ===================================================== pushd C:\h\w\AFB00942\w\BE220A24\e\
"C:\h\w\AFB00942\p\dotnet.exe" exec --runtimeconfig System.Text.Encodings.Web.Tests.runtimeconfig.json --depsfile System.Text.Encodings.Web.Tests.deps.json xunit.console.dll System.Text.Encodings.Web.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing popd
===========================================================================================================
C:\h\w\AFB00942\w\BE220A24\e>"C:\h\w\AFB00942\p\dotnet.exe" exec --runtimeconfig System.Text.Encodings.Web.Tests.runtimeconfig.json --depsfile System.Text.Encodings.Web.Tests.deps.json xunit.console.dll System.Text.Encodings.Web.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing Discovering: System.Text.Encodings.Web.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Text.Encodings.Web.Tests (found 212 test cases)
Starting: System.Text.Encodings.Web.Tests (parallel test collections = on, max threads = 2)
System.Text.Encodings.Web.Tests.AllowedBmpCodePointsBitmapTests.ForbidUndefinedCharacters_RemovesUndefinedChars [FAIL]
Assert.Equal() Failure
Expected: True
Actual: False
Stack Trace:
/_/src/libraries/System.Text.Encodings.Web/tests/AllowedBmpCodePointsBitmapTests.cs(87,0): at System.Text.Encodings.Web.Tests.AllowedBmpCodePointsBitmapTests.ForbidUndefinedCharacters_RemovesUndefinedChars()
Finished: System.Text.Encodings.Web.Tests
=== TEST EXECUTION SUMMARY ===
System.Text.Encodings.Web.Tests Total: 652, Errors: 0, Failed: 1, Skipped: 0, Time: 4.336s
----- end Mon 12/06/2021 15:46:23.04 ----- exit code 1 ----------------------------------------------------------
2021-12-06T15:46:23.715Z	INFO run.py	run(48)	main	Beginning reading of test results.
2021-12-06T15:46:23.715Z	INFO run.py	__init__(48)	get_log_files	Searching 'C:\h\w\AFB00942\w\BE220A24\e\..' for log files
Found log 'C:\h\w\AFB00942\w\BE220A24\e\..\console.95c30e0d.log'
Uri 'https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-jajahoda-libmatrix-5e681f823efa4a3e8b/System.Text.Encodings.Web.Tests/console.95c30e0d.log?sv=2019-07-07&se=2021-12-26T15%3A34%3A41Z&sr=c&sp=rl&sig=OL7%2FjCWh8CqpIwnNF0vcXSssUrDYhx85alvT4QDHCqg%3D'
2021-12-06T15:46:23.715Z	INFO run.py	__init__(66)	construct_log_list	Generated log list: console.95c30e0d.log:
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-jajahoda-libmatrix-5e681f823efa4a3e8b/System.Text.Encodings.Web.Tests/console.95c30e0d.log?sv=2019-07-07&se=2021-12-26T15%3A34%3A41Z&sr=c&sp=rl&sig=OL7%2FjCWh8CqpIwnNF0vcXSssUrDYhx85alvT4QDHCqg%3D
2021-12-06T15:46:23.715Z	INFO run.py	__init__(90)	read_results	Searching 'C:\h\w\AFB00942\w\BE220A24\e' for test results files
2021-12-06T15:46:23.715Z	INFO run.py	__init__(96)	read_results	Found results file C:\h\w\AFB00942\w\BE220A24\e\testResults.xml with format xunit
2021-12-06T15:46:23.715Z	INFO run.py	__init__(90)	read_results	Searching 'C:\h\w\AFB00942\w\BE220A24\uploads' for test results files
2021-12-06T15:46:23.715Z	INFO run.py	packing_test_reporter(30)	report_results	Packing 652 test reports to 'C:\h\w\AFB00942\w\BE220A24\e\__test_report.json'
2021-12-06T15:46:23.731Z	INFO run.py	packing_test_reporter(33)	report_results	Packed 189433 bytes
Did not find dumps, skipping dump docs generation.

danmoseley · 2022-03-23T03:57:17Z

@agocke is this good to merge? i fixed a conflict

danmoseley · 2022-03-23T03:57:42Z

not sure whether I need to trigger any extra legs?

danmoseley · 2022-03-27T22:43:45Z

@agocke ?

failure is only #65817 but maybe other legs need to be triggered?

agocke · 2022-03-28T07:36:49Z

Sorry, not sure why I missed this before. LGTM.

* CI matrix change: Windows * Remove Windows Server RS5 * Revert .net Framework change * Add 19H1 to rolling * Add Windows.Server.Core.20H2 to CI matrix * Remove Windows 8.1 * Remove 20H2 as thhere is a PR for it dotnet#60054 * Typo Co-authored-by: Jan Jahoda <jajahoda@microsoft.com> Co-authored-by: Andy Gocke <andy@commentout.net> Co-authored-by: Dan Moseley <danmose@microsoft.com>

CI matrix change: Windows
366ff58

This was referenced Sep 11, 2021
Update unsupported distros in helix queues to latest and add missing ones #57947
Closed
add W11 test queue #58570
Merged

aik-jahoda requested review from a team and wfurt September 13, 2021 07:26

jkoritzinsky reviewed Sep 13, 2021
View reviewed changes

Comment threadeng/pipelines/coreclr/templates/helix-queues-setup.yml

wfurt reviewed Sep 13, 2021
View reviewed changes

Comment threadeng/pipelines/libraries/helix-queues-setup.yml Outdated

Jan Jahoda added 3 commits September 14, 2021 17:42

Remove Windows Server RS5
3b0f94e

Merge remote-tracking branch 'upstream/main' into jajahoda/libmatrix-…
b359267
…windows

Revert .net Framework change
a165492

safern reviewed Sep 21, 2021
View reviewed changes

Add 19H1 to rolling
89fbd40

aik-jahoda requested review from jkoritzinsky and safern September 24, 2021 15:03

Add Windows.Server.Core.20H2 to CI matrix
492a6f2

runfoappBot mentioned this pull request Sep 24, 2021
System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix Failing in CI #59541
Closed

Jan Jahoda added 3 commits September 29, 2021 15:04

Merge remote-tracking branch 'upstream/main' into jajahoda/libmatrix-…
44bb535
…windows

Remove Windows 8.1
2aa6e36

Merge remote-tracking branch 'origin/jajahoda/libmatrix-windows' into…
9e54a1a
… jajahoda/libmatrix-windows

aik-jahoda requested review from a team and jkotas September 30, 2021 13:58

safern reviewed Sep 30, 2021
View reviewed changes

safern approved these changes Oct 1, 2021
View reviewed changes

jkoritzinsky approved these changes Oct 1, 2021
View reviewed changes

Merge remote-tracking branch 'upstream/main' into jajahoda/libmatrix-…
3bb28e7
…windows

runfoappBot mentioned this pull request Oct 5, 2021
System.Net.Sockets.Tests.SendPacketsAsync.SendPacketsElement_FileZeroCount_Success sometimes fails #60017
Closed

marek-safar added the area-Infrastructure label Nov 1, 2021

ericstj assigned aik-jahoda Nov 1, 2021

Merge remote-tracking branch 'upstream/main' into jajahoda/libmatrix-…
f7ba705
…windows

aik-jahoda mentioned this pull request Nov 23, 2021
[Don't merge CI test only] #61970
Closed

Remove 20H2 as thhere is a PR for itdotnet#60054
c0dec52

agockeand others added 3 commits March 6, 2022 22:32

Merge branch 'main' into jajahoda/libmatrix-windows
0f2e65f

Typo
4007922

Merge branch 'main' into jajahoda/libmatrix-windows
1e9b9ff

danmoseley unassigned aik-jahoda Mar 27, 2022

agocke merged commit bd43f55 into dotnet:mainMar 28, 2022

ghost locked as resolved and limited conversation to collaborators Apr 27, 2022

Uh oh!

Conversation

aik-jahoda commented Sep 11, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Sep 11, 2021

Uh oh!

Uh oh!

wfurt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aik-jahoda commented Sep 29, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ghost commented Nov 1, 2021

Uh oh!

karelz commented Nov 29, 2021

Uh oh!

aik-jahoda commented Dec 8, 2021

Uh oh!

aik-jahoda commented Dec 14, 2021

Uh oh!

danmoseley commented Mar 23, 2022

Uh oh!

danmoseley commented Mar 23, 2022

Uh oh!

danmoseley commented Mar 27, 2022

Uh oh!

agocke commented Mar 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

aik-jahoda commented Sep 11, 2021•
edited
Loading