Remove pinned XunitAnalyzersVersion from Versions.props by akoeplinger · Pull Request #118241 · dotnet/runtime

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

akoeplinger · 2025-07-31T15:46:58Z

Addresses all the new warnings in the newer analyzer version.

Fixes#97088

…ed by the parameters of the test method

…urces

…recated in xUnit.net v3

… results

…xist in a collection

Copilot

Pull Request Overview

This PR addresses fixes for XUnit analyzer issues by updating test method signatures to use nullable parameter types where null values are being tested. The changes ensure that test parameter declarations are correctly annotated as nullable when the test methods are designed to accept null values through inline data attributes.

Updates test method parameter types to use nullable annotations (e.g., string?, Type?, etc.)
Fixes test methods to properly handle async/await patterns where incorrect return types were used
Includes minor project file updates to include necessary test infrastructure files

Reviewed Changes

Copilot reviewed 299 out of 377 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/Reflection/AssemblyMetadataAttributeTests.cs	Updated test parameter to accept nullable string values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/NullableTests.cs	Updated test parameter to accept nullable Type values
src/libraries/System.Runtime/tests/System.Runtime.Tests/System/IntPtrTests.cs	Updated test parameters to accept nullable object values
src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs	Fixed async test methods to return Task instead of void
src/libraries/System.Runtime.InteropServices.JavaScript/tests/System.Runtime.InteropServices.JavaScript.UnitTests/System/Runtime/InteropServices/JavaScript/JSExportTest.cs	Fixed async test method return type
src/libraries/System.IO.FileSystem.Tests/File/AppendAllBytesAsync.cs	Fixed async test method to properly await operations
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/FileSystemTest.cs	Updated TheoryData generic types for better type safety
src/libraries/System.Linq/tests/WhereTests.cs	Fixed test assertions to use appropriate methods
Many other test files	Similar updates to parameter nullability annotations and async patterns

Comments suppressed due to low confidence (1)

src/libraries/System.Net.Sockets/tests/FunctionalTests/DisconnectTest.cs:76

This ActiveIssue attribute was added but there's no context about why this test method was marked as having an active issue. The GitHub issue reference should be validated to ensure it's correctly tracking the actual problem.

 [ActiveIssue("https://github.com/dotnet/runtime/issues/118314")]

* Remove pinned XunitAnalyzersVersion from Versions.props Fixesdotnet#97088 * Fix xUnit1012: Null should not be used for value type parameters * Fix xUnit1037: There are fewer theory data type arguments than required by the parameters of the test method * Fix xUnit1041: Fixture arguments to test classes must have fixture sources * Fix xUnit1048: Avoid using 'async void' for test methods as it is deprecated in xUnit.net v3 * Fix xUnit2021: Async assertions should be awaited * Fix xUnit2027: Comparison of sets to linear containers have undefined results * Fix xUnit2029: Do not use Assert.Empty to check if a value does not exist in a collection * Fix xUnit2031: Do not use Where clause with Assert.Single * Disable test that started erroring

MichalStrehovsky · 2025-08-05T11:35:22Z

I'm seeing coreCLR pri-1 outerloop failures that look like:

src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.cs(56,30): error xUnit1041: Fixture argument 'id' does not have a fixture source (if it comes from a collection definition, ensure the definition is in the same assembly as the test) (https://xunit.net/xunit.analyzers/rules/xUnit1041) [/__w/1/s/src/tests/baseservices/exceptions/regressions/V1/SEH/COOL/rethrow.csproj]

E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1114184&view=logs&jobId=71d923da-3b2b-5516-1160-0ea57864003b&j=71d923da-3b2b-5516-1160-0ea57864003b&t=9df8b2f7-960f-560c-e71f-0a84b98b0ae3

Could this be related to the upgraded xunit analyzers?

akoeplinger · 2025-08-05T12:19:12Z

yes. I'll fix it

Remove pinned XunitAnalyzersVersion from Versions.props
380ff2e
Fixes#97088

github-actionsBot added the area-codeflow for labeling automated codeflow label Jul 31, 2025

dotnet-policy-serviceBot assigned akoeplinger Jul 31, 2025

This was referenced Jul 31, 2025
slow macOS - "##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 60 minutes." dotnet/dnceng#1883
Open
The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008
Open

akoeplinger force-pushed the akoeplinger-patch-2 branch from eca1e64 to df2b47fCompare August 1, 2025 22:03

This was referenced Aug 2, 2025
TimeZoneInfoTests.NoBackwardTimeZones tests are failing on Android #117731
Open
[release/8.0] TimeZoneInfoTests.Platform_TimeZoneNames failing on Android #117903
Open

akoeplinger added 2 commits August 2, 2025 14:06

Fix xUnit1012: Null should not be used for value type parameters
483c77d

Fix xUnit1037: There are fewer theory data type arguments than requir…
1597e3b
…ed by the parameters of the test method

akoeplinger force-pushed the akoeplinger-patch-2 branch from df2b47f to 8e49b74Compare August 2, 2025 14:07

Fix xUnit1041: Fixture arguments to test classes must have fixture so…
54561d9
…urces

akoeplinger force-pushed the akoeplinger-patch-2 branch from 8e49b74 to 3d6b698Compare August 2, 2025 22:05

This was referenced Aug 3, 2025
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. dotnet/dnceng#5922
Open
System.Net.Quic.Tests failed with "System.Net.Quic.QuicException : The connection timed out from inactivity." #105177
Closed

akoeplinger mentioned this pull request Aug 3, 2025
Disconnect_NotConnected_ThrowsInvalidOperationException throws wrong exception #118314
Closed

akoeplinger force-pushed the akoeplinger-patch-2 branch from 3d6b698 to c340a3cCompare August 3, 2025 10:56

akoeplinger added 3 commits August 3, 2025 14:54

Fix xUnit1048: Avoid using 'async void' for test methods as it is dep…
24a149a
…recated in xUnit.net v3

Fix xUnit2021: Async assertions should be awaited
443b396

Fix xUnit2027: Comparison of sets to linear containers have undefined…
41a2067
… results

akoeplinger force-pushed the akoeplinger-patch-2 branch from c340a3c to 9c7794eCompare August 3, 2025 12:54

akoeplinger added 3 commits August 3, 2025 16:02

Fix xUnit2029: Do not use Assert.Empty to check if a value does not e…
c61cff7
…xist in a collection

Fix xUnit2031: Do not use Where clause with Assert.Single
f87131b

Disable test that started erroring
a6b34ad

akoeplinger force-pushed the akoeplinger-patch-2 branch from 9c7794e to a6b34adCompare August 3, 2025 14:03

This was referenced Aug 3, 2025
[Long Running Test] 'System.Composition.UnitTests.ConcurrencyTests.SharedInstancesAreNotVisibleUntilActivationCompletes' #113686
Open
System.ComponentModel.TypeConverter.Tests.WorkItemExecution failure #117961
Closed

akoeplinger marked this pull request as ready for review August 4, 2025 08:15

CopilotAI review requested due to automatic review settings August 4, 2025 08:15

akoeplinger requested review from ilonatommy, lewing, maraf and pavelsavara as code owners August 4, 2025 08:15

CopilotAI reviewed Aug 4, 2025
View reviewed changes

Comment threadsrc/libraries/System.Linq/tests/WhereTests.cs
Comment threadsrc/libraries/System.Linq/tests/WhereTests.cs

ViktorHofer approved these changes Aug 4, 2025
View reviewed changes

Merge branch 'main' into akoeplinger-patch-2
7014441

akoeplinger enabled auto-merge (squash) August 4, 2025 18:33

akoeplinger merged commit a44f532 into mainAug 4, 2025
150 of 152 checks passed

build-analysisBot mentioned this pull request Aug 4, 2025
Test failure: System.Reflection.Metadata.ApplyUpdateTest.TestGenericAddStaticField #115318
Closed

akoeplinger deleted the akoeplinger-patch-2 branch August 5, 2025 12:19

jakobbotsch mentioned this pull request Aug 5, 2025
Fix illumos compile errors in coreclr/tools/superpmi/mcs/verbmerge.cpp #117464
Merged

jkotas mentioned this pull request Aug 5, 2025
Pri1 CoreCLR test build is failing with xUnit1041 error #118409
Closed

github-actionsBot locked and limited conversation to collaborators Sep 5, 2025