Skip to content

Enable parallelism in test projects - #5697

Merged
danielmarbach merged 17 commits into
masterfrom
rhys/test-parallel
Aug 7, 2026
Merged

Enable parallelism in test projects#5697
danielmarbach merged 17 commits into
masterfrom
rhys/test-parallel

Conversation

@rbev

@rbevrbev commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces several improvements to the test infrastructure to introduce parallelisation within the test projects to improve CI build times.

Test Infrastructure and Parallelization Enhancements:

  • Added global NUnit settings (FixtureLifeCycle, Parallelizable, and LevelOfParallelism) to both src/Particular.LicensingComponent.UnitTests/NUnitSettings.cs and src/ServiceControl.AcceptanceTests.RavenDB/NUnitSettings.cs to enable test parallelization and control concurrency level.
  • Introduced a SetupFixture in src/ServiceControl.AcceptanceTests.RavenDB/SetupFixture.cs to ensure required event log sources are created before tests run, preventing race conditions during test startup.
  • Added a static SemaphoreSlim-based lock and UseDatabaseLifecycleLock method in AcceptanceTestStorageConfiguration to serialize database lifecycle operations, preventing parallel access issues with the shared RavenDB server.

Test Safety and Reliability:

  • Marked tests that interact with shared resources (e.g., CORS, forwarded headers, critical errors, message import failures) as [NonParallelizable] to avoid race conditions and ensure reliable test execution.

Groups with full parallelism

GroupTest Projects
AzureStorageQueuesServiceControl.Transports.ASQ.Tests
DefaultServiceControl.Audit.AcceptanceTests, ServiceControl.Audit.AcceptanceTests.RavenDB (linked NUnitSettings.cs from Audit.AcceptanceTests), ServiceControl.Audit.Persistence.Tests, ServiceControl.Audit.Persistence.Tests.RavenDB (linked NUnitSettings.cs from Audit.Persistence.Tests), ServiceControl.Audit.UnitTests, ServiceControl.Config.Tests, ServiceControl.Infrastructure.Tests, ServiceControlInstaller.Engine.UnitTests, ServiceControlInstaller.Packaging.UnitTests, ServiceControl.Monitoring.AcceptanceTests, ServiceControl.Monitoring.UnitTests, ServiceControl.MultiInstance.AcceptanceTests, ServiceControl.Persistence.Tests.InMemory, ServiceControl.Transports.Tests, ServiceControl.UnitTests, Particular.LicensingComponent.UnitTests
IBMMQServiceControl.Transports.IBMMQ.Tests
PostgreSQLServiceControl.Transports.PostgreSql.Tests
PostgreSQLPersistenceServiceControl.Persistence.Tests.PostgreSql
PrimaryRavenAcceptanceServiceControl.AcceptanceTests.RavenDB
PrimaryRavenPersistenceServiceControl.Persistence.Tests.RavenDB
SqlServerPersistenceServiceControl.Persistence.Tests.SqlServer

Groups not updated

GroupTest Projects
AzureServiceBusServiceControl.Transports.ASBS.Tests
MSMQServiceControl.Transports.Msmq.Tests
RabbitMQServiceControl.Transports.RabbitMQClassicConventionalRouting.Tests, ServiceControl.Transports.RabbitMQClassicDirectRouting.Tests, ServiceControl.Transports.RabbitMQQuorumConventionalRouting.Tests, ServiceControl.Transports.RabbitMQQuorumDirectRouting.Tests
SQSServiceControl.Transports.SQS.Tests
SqlServerServiceControl.Transports.SqlServer.Tests

@rbev
rbevforce-pushed the rhys/test-parallel branch from 7bac798 to f747755CompareAugust 4, 2026 08:03
@rbev
rbev marked this pull request as draft August 4, 2026 08:32
@rbev
rbevforce-pushed the rhys/test-parallel branch from 28b6e7d to 20f562dCompareAugust 5, 2026 06:08
@rbev
rbev marked this pull request as ready for review August 5, 2026 08:04
@johnsimons

Copy link
Copy Markdown
Member

Should we just use the default for LevelOfParallelismhttps://docs.nunit.org/articles/nunit/writing-tests/attributes/levelofparallelism.html, which mean not using the attribute?

@rbev

rbev commented Aug 6, 2026

Copy link
Copy Markdown
ContributorAuthor

Should we just use the default for LevelOfParallelismhttps://docs.nunit.org/articles/nunit/writing-tests/attributes/levelofparallelism.html, which mean not using the attribute?

I removed the explicit setting and it ran much slower, the build agents don't have 4 cores and benefit from having higher parallelism.

Comment threadsrc/ServiceControl.Transports.Tests/Learning/TransportTestsConfiguration.cs Outdated
@danielmarbach

Copy link
Copy Markdown
Contributor

Curious did you already compare before and after? How much gain do we get?

@rbev

rbev commented Aug 7, 2026

Copy link
Copy Markdown
ContributorAuthor

Curious did you already compare before and after? How much gain do we get?

Here's the groupings that have changed, omitting some because they were already parallel from a previous pr, it's based on a single run so take it with a grain of salt.

GroupParallelismBefore-LinuxBefore-WindowsAfter-LinuxAfter-Windows
AzureStorageQueues✅ Enabled4m 30s6m 27s4m 33s6m 58s
Default✅ Enabled10m 47s13m 40s7m 47s11m 17s
IBMMQ✅ Enabled3m 56s3m 38s
PostgreSQL✅ Enabled2m 52s7m 45s3m 47s7m 23s
PrimaryRavenAcceptance✅ Enabled10m 51s14m 17s6m 31s9m 49s
PrimaryRavenPersistence✅ Enabled4m 0s8m 23s5m 7s7m 34s

@danielmarbachdanielmarbach left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@danielmarbach
danielmarbach merged commit 6cbabe2 into masterAug 7, 2026
71 of 72 checks passed
@danielmarbach
danielmarbach deleted the rhys/test-parallel branch August 7, 2026 04:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@rbev@johnsimons@danielmarbach