Uh oh!
There was an error while loading. Please reload this page.
Significantly speed up BeEquivalentTo for large unordered collections - #3188
Conversation
Qodana for .NETIt seems all right 👌 No new problems were found according to the checks applied 💡 Qodana analysis was run in the pull request mode: only the changed files were checked Contact Qodana teamContact us at qodana-support@jetbrains.com
|
9973452 to
75ccee2Compared141c65 to
e0e1ef5Compare…collections Replace the O(n! * n) permutation search in FindClosestMismatches with a two-strategy approach: - For n <= 8 items remaining, keep the exact permutation search (globally optimal; factorial(8) = 40,320, well within cost). - For n > 8, use a greedy O(n^2 log n) assignment that sorts all n^2 (expectation, subject) pairs by failure count and picks the closest unassigned pair first. Ties are broken by expectation index then subject index to preserve deterministic, natural-order error messages. Also extract ReferentialComparer to its own file and bump Reflectify to 1.9.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AsCollectionItem<T>(int) now returns a pre-computed string for indices 0-1023 instead of calling ToString() on every comparison. For large collections this avoids repeated short-lived string allocations in hot paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add UseDryRun flag to AssertionScope. When set, AssertionChain.FailWith() increments a counter instead of formatting and storing the full message. LooselyOrderedEquivalencyStrategy uses a dry-run scope in TryToMatchCount to count failures cheaply, split caches (countCache for counts, fullFailuresCache for strings), and updates FindClosestMismatches to take separate getFailureCount and getFullFailures delegates — deferring string allocation to the winning assignment only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…k overflows Dry-run comparisons in TryToMatchCount and TryToMatch share the parent context's CyclicReferenceDetector by default. When the same object appears in multiple positions in an unordered collection, the detector could flag subsequent dry-runs as cyclic references and skip them — producing wrong failure counts and potential stack overflows. Fix by cloning the detector before each dry-run invocation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GetFailureCount() was only counting direct RegisterFailure() calls. Failures from nested assertion scopes (e.g. via AssertionRuleEquivalencyStep) are stored in assertionStrategy.FailureMessages rather than the local counter, so they were invisible to TryToMatchCount — causing mismatches to look like matches (count=0) and corrupting the optimal assignment. Also call scope.Discard() after GetFailureCount() so that the dry-run scope does not propagate its failure messages to the parent scope on disposal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
e0e1ef5 to
4883856Compare
jnyrup
left a comment
There was a problem hiding this comment.
Looks good - and what a pleasure to review with the focused commits
Did you do a benchmark on Compare_complex_collection to get before and after cpu/mem numbers?
For later work, I'm wondering if we might be able to utilize UseDryRun more places?
Uh oh!
There was an error while loading. Please reload this page.
dennisdoomen
commented
Apr 12, 2026
I'm using Copilot CLI heavily, so thanks to Copilot ;-)
Most likely. But haven't investigated that yet. |
dennisdoomen
commented
Apr 12, 2026
Added all the benchmarks. |
The strategy already owns TryToMatchCount and TryToMatch, so keeping the closest-match helper chain static forced delegate plumbing without adding value. Make those helpers instance methods so they can use the existing instance members directly while preserving the matching logic, scoring, and caches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Updated [FluentAssertions](https://github.com/fluentassertions/fluentassertions) from 8.2.0 to 8.10.0. <details> <summary>Release notes</summary> _Sourced from [FluentAssertions's releases](https://github.com/fluentassertions/fluentassertions/releases)._ ## 8.10.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### Improvements * Fail with a descriptive error when path-based rules are used on value-semantic types by @dennisdoomen in fluentassertions/fluentassertions#3187 * Significantly speed up BeEquivalentTo for large unordered collections by @dennisdoomen in fluentassertions/fluentassertions#3188 * Add ComparingNullCollectionsAsEmpty and ComparingNullStringsAsEmpty options to BeEquivalentTo by @dennisdoomen in fluentassertions/fluentassertions#3202 * Include original index in extraneous item failure messages by @dennisdoomen in fluentassertions/fluentassertions#3203 ### Documentation * Reroute the docs link to Xceed by @dennisdoomen in fluentassertions/fluentassertions#3183 * Fix typo in release notes by @jnyrup in fluentassertions/fluentassertions#3194 * Fix typos in docs by @jnyrup in fluentassertions/fluentassertions#3197 ### Others * Bump flatted from 3.4.1 to 3.4.2 in the npm_and_yarn group across 1 directory by @dependabot[bot] in fluentassertions/fluentassertions#3184 * Add AI assistant instruction file (agents.md) for Copilot, Claude, and JetBrains Junie by @Copilot in fluentassertions/fluentassertions#3176 * Bump smol-toml from 1.6.0 to 1.6.1 in the npm_and_yarn group across 1 directory by @dependabot[bot] in fluentassertions/fluentassertions#3185 * Bump the npm_and_yarn group across 1 directory with 2 updates by @dependabot[bot] in fluentassertions/fluentassertions#3186 * Bump cspell from 9.7.0 to 10.0.0 by @dependabot[bot] in fluentassertions/fluentassertions#3189 * Update nugets by @jnyrup in fluentassertions/fluentassertions#3192 * Fixup Qodana issues by @jnyrup in fluentassertions/fluentassertions#3193 * Fix Qodana argument separator by @jnyrup in fluentassertions/fluentassertions#3195 * Use new Qodana linter option by @jnyrup in fluentassertions/fluentassertions#3196 * Fix flaky BeLessThanOrEqualTo execution time test by @Copilot in fluentassertions/fluentassertions#3200 * Bump JetBrains/qodana-action from 2025.3 to 2026.1 by @dependabot[bot] in fluentassertions/fluentassertions#3201 * Use long for hashCode in ReferentialComparer to avoid overflow by @dennisdoomen in fluentassertions/fluentassertions#3204 **Full Changelog**: fluentassertions/fluentassertions@8.9.0...8.10.0 ## 8.9.0 <!-- Release notes generated using configuration in .github/release.yml at main --> > [!WARNING] > For projects targeting .NET 9, you need at least .NET SDK 9.0.200 as earlier versions will trigger compile errors because of invalid overload resolution. See #3225 ## What's Changed ### New features * Add support for `Span<T>`, `ReadOnlySpan<T>`, `Memory<T>` and `ReadOnlyMemory<T>` by @dennisdoomen in fluentassertions/fluentassertions#3172 ### Improvements * Allow excluding all properties by type from `BeEquivalentTo` by @Copilot in fluentassertions/fluentassertions#3115 * Clean-up the stack trace when throwing an assertion failure by @dennisdoomen in fluentassertions/fluentassertions#3152 * Improve reporting the differences between differently sized collections in `BeEquivalentTo` by @dennisdoomen in fluentassertions/fluentassertions#3133 * Improve reporting the subject when chaining `Throw` and `Which` by @dennisdoomen in fluentassertions/fluentassertions#3160 * Add `HaveMillisecond`/`NotHaveMillisecond` assertion methods for `DateTime` and `DateTimeOffset` by @Copilot in fluentassertions/fluentassertions#3164 * Add `BeEqualTo` and `NotBeEqualTo` as collection assertion aliases by @Copilot in fluentassertions/fluentassertions#3166 ### Fixes * Fix formatting exception when comparing strings containing braces by @Copilot in fluentassertions/fluentassertions#3151 ### Documentation * Also mention the global configuration options in the docs. by @dennisdoomen in fluentassertions/fluentassertions#3132 * Add xUnit migration tips by @fuguiKz in fluentassertions/fluentassertions#3141 * Point the docs to the new site by @dennisdoomen in fluentassertions/fluentassertions#3155 * Added missing release notes by @dennisdoomen in fluentassertions/fluentassertions#3161 * Fix "an" vs "a" typos by @jnyrup in fluentassertions/fluentassertions#3174 ### Others * Bump actions/download-artifact from 5 to 6 by @dependabot[bot] in fluentassertions/fluentassertions#3116 * Bump cspell from 9.2.1 to 9.2.2 by @dependabot[bot] in fluentassertions/fluentassertions#3118 * Bump actions/upload-artifact from 4 to 5 by @dependabot[bot] in fluentassertions/fluentassertions#3117 * Add NuGet package attestations using GitHub provenance by @Copilot in fluentassertions/fluentassertions#3119 * Bump cspell from 9.2.2 to 9.3.0 by @dependabot[bot] in fluentassertions/fluentassertions#3122 * Bump actions/attest-build-provenance from 2 to 3 by @dependabot[bot] in fluentassertions/fluentassertions#3121 * fix qodana warnings by @jnyrup in fluentassertions/fluentassertions#3123 * Update nuget packages by @jnyrup in fluentassertions/fluentassertions#3124 * Bump cspell from 9.3.0 to 9.3.2 by @dependabot[bot] in fluentassertions/fluentassertions#3125 * Bump actions/checkout from 5 to 6 by @dependabot[bot] in fluentassertions/fluentassertions#3127 * Update to NET 10 SDK by @jnyrup in fluentassertions/fluentassertions#3128 * Use `==` or `!=` when comparing Nullable<T> against constants by @jnyrup in fluentassertions/fluentassertions#3129 * Create polyfill for `string.Create` by @jnyrup in fluentassertions/fluentassertions#3130 * Bump cspell from 9.3.2 to 9.4.0 by @dependabot[bot] in fluentassertions/fluentassertions#3131 * Bump actions/cache from 4 to 5 by @dependabot[bot] in fluentassertions/fluentassertions#3136 * Bump actions/upload-artifact from 5 to 6 by @dependabot[bot] in fluentassertions/fluentassertions#3135 * Bump actions/download-artifact from 6 to 7 by @dependabot[bot] in fluentassertions/fluentassertions#3134 * Cleanups by @jnyrup in fluentassertions/fluentassertions#3137 * Nuget updates by @jnyrup in fluentassertions/fluentassertions#3139 * Suppress `UnassignedGetOnlyAutoProperty` for `Node.GetHashCode` by @jnyrup in fluentassertions/fluentassertions#3138 * Use `NonReadonlyMemberInGetHashCode` by @jnyrup in fluentassertions/fluentassertions#3140 * Bump JetBrains/qodana-action from 2025.2 to 2025.3 by @dependabot[bot] in fluentassertions/fluentassertions#3142 * Use compiler-generated `paramName` parameter by @jnyrup in fluentassertions/fluentassertions#3143 * Fix `When_concurrently_getting_equality_strategy_it_should_not_throw` by @jnyrup in fluentassertions/fluentassertions#3144 * Bump cspell from 9.4.0 to 9.6.0 by @dependabot[bot] in fluentassertions/fluentassertions#3146 * Bump cspell from 9.6.0 to 9.6.2 by @dependabot[bot] in fluentassertions/fluentassertions#3153 ... (truncated) ## 8.8.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### New features * Add support for MSTest4 by @jnyrup in fluentassertions/fluentassertions#3111 ### Improvements * Allow WithoutMessage when using Should().Throw() and ThrowAsync() by @dennisdoomen in fluentassertions/fluentassertions#3100 * Improve reporting of the differences between long strings by @dennisdoomen in fluentassertions/fluentassertions#3101 * Allow customizing the equivalency behavior for BeXmlSerializable by @logiclrd in fluentassertions/fluentassertions#3107 ### Documentation * Add docs for `config` parameter by @jnyrup in fluentassertions/fluentassertions#3104 ### Others * Refreshed readme by @dennisdoomen in fluentassertions/fluentassertions#3106 * Merge hotfix 8.7.1 back to main by @dennisdoomen in fluentassertions/fluentassertions#3109 * Fix release notes by @jnyrup in fluentassertions/fluentassertions#3112 * Bump github/codeql-action from 3 to 4 by @dependabot[bot] in fluentassertions/fluentassertions#3113 **Full Changelog**: fluentassertions/fluentassertions@8.7.1...8.8.0 ## 8.7.1 <!-- Release notes generated using configuration in .github/release.yml at hotfix/8.7.1 --> ## What's Changed ### Others * JSON assertions did not properly handle floats, doubles and unsigned … by @dennisdoomen in fluentassertions/fluentassertions#3105 * Fixed ambiguity when using Should on a JsonNode derived class ... by @JSkimming in fluentassertions/fluentassertions#3102 **Full Changelog**: fluentassertions/fluentassertions@8.7.0...8.7.1 ## 8.7.0 <!-- Release notes generated using configuration in .github/release.yml at ae620add07cf6666841e568fd3bf8a0733478bb5 --> ## What's Changed ### New features * Added System.Text.Json assertion APIs and BeEquivalentTo support by @dennisdoomen in fluentassertions/fluentassertions#3094 ### Others * Address a bunch of Qodana issues by @dennisdoomen in fluentassertions/fluentassertions#3082 * Bump actions/checkout from 4 to 5 by @dependabot[bot] in fluentassertions/fluentassertions#3085 * Bump JetBrains/qodana-action from 2025.1 to 2025.2 by @dependabot[bot] in fluentassertions/fluentassertions#3086 * Bump actions/download-artifact from 4 to 5 by @dependabot[bot] in fluentassertions/fluentassertions#3087 * Bump cspell from 9.2.0 to 9.2.1 by @dependabot[bot] in fluentassertions/fluentassertions#3090 * Bump actions/setup-dotnet from 4 to 5 by @dependabot[bot] in fluentassertions/fluentassertions#3092 * Add lock file for nuget packages by @jnyrup in fluentassertions/fluentassertions#3084 * Set `DisableImplicitNuGetFallbackFolder` by @jnyrup in fluentassertions/fluentassertions#3095 * Bump Nugets by @jnyrup in fluentassertions/fluentassertions#3096 * Revert package locking by @jnyrup in fluentassertions/fluentassertions#3098 * Clean-up tests related to exceptions by @dennisdoomen in fluentassertions/fluentassertions#3099 **Full Changelog**: fluentassertions/fluentassertions@8.6.0...8.7.0 ## 8.6.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### Improvements * Add support for inline assertions using `Value.ThatMatches` and `Value.ThatSatisfies` by @dennisdoomen in fluentassertions/fluentassertions#3076 ### Others * Remove Microsoft.SourceLink.GitHub by @SimonCropp in fluentassertions/fluentassertions#3072 * Bump cspell from 9.1.3 to 9.1.5 by @dependabot[bot] in fluentassertions/fluentassertions#3073 * Added PackageGuard to the build pipeline by @dennisdoomen in fluentassertions/fluentassertions#3075 * Bump cspell from 9.1.5 to 9.2.0 by @dependabot[bot] in fluentassertions/fluentassertions#3077 * Remove dependencies on Bogus by @jnyrup in fluentassertions/fluentassertions#3080 * Update nuget packages by @jnyrup in fluentassertions/fluentassertions#3081 * Follow-up to #3076 by @jnyrup in fluentassertions/fluentassertions#3079 * Documentation and typo fixes by @jnyrup in fluentassertions/fluentassertions#3078 ## New Contributors * @SimonCropp made their first contribution in fluentassertions/fluentassertions#3072 **Full Changelog**: fluentassertions/fluentassertions@8.5.0...8.6.0 ## 8.5.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### New features * Extended BeEquivalentTo with support for checking the type of the properties by @dennisdoomen in fluentassertions/fluentassertions#3066 ### Fixes * Ensured WithTracing is safe when used with BeEquivalentTo globally by @dennisdoomen in fluentassertions/fluentassertions#3067 ### Others * Revisit the .editorconfig rules by @dennisdoomen in fluentassertions/fluentassertions#3063 * Bump cspell from 9.1.1 to 9.1.2 by @dependabot[bot] in fluentassertions/fluentassertions#3068 * Use .NET 9 SDK by @jnyrup in fluentassertions/fluentassertions#3069 * Bump cspell from 9.1.2 to 9.1.3 by @dependabot[bot] in fluentassertions/fluentassertions#3071 **Full Changelog**: fluentassertions/fluentassertions@8.4.0...8.5.0 ## 8.4.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### Improvements * Allow excluding properties by name anywhere in the graph by @dennisdoomen in fluentassertions/fluentassertions#3062 * Added Should().Throw(), ThrowAsync() and ThrowWithinAsync() flavors that don’t require a specific exception type by @Xceed-DelvaJB in fluentassertions/fluentassertions#3059 ### Others * Bump cspell from 9.0.1 to 9.0.2 by @dependabot in fluentassertions/fluentassertions#3050 * Bump JetBrains/qodana-action from 2024.2 to 2025.1 by @dependabot in fluentassertions/fluentassertions#3044 * Fix qodana warnings by @jnyrup in fluentassertions/fluentassertions#3053 * Add contributor grant by @dennisdoomen in fluentassertions/fluentassertions#3055 * Bump cspell from 9.0.2 to 9.1.1 by @dependabot in fluentassertions/fluentassertions#3058 ## New Contributors * @Xceed-DelvaJB made their first contribution in fluentassertions/fluentassertions#3059 **Full Changelog**: fluentassertions/fluentassertions@8.3.0...8.4.0 ## 8.3.0 <!-- Release notes generated using configuration in .github/release.yml at main --> ## What's Changed ### Improvements * Improve rendering of exception messages when using WithMessage by @dennisdoomen in fluentassertions/fluentassertions#3039 * Clarify the date/time type when comparing dates, times and combinations of those by @dennisdoomen in fluentassertions/fluentassertions#3049 ### Others * Bump cspell from 8.17.5 to 8.18.1 by @dependabot in fluentassertions/fluentassertions#3041 * Bump cspell from 8.18.1 to 8.19.2 by @dependabot in fluentassertions/fluentassertions#3042 * Bump cspell from 8.19.2 to 8.19.3 by @dependabot in fluentassertions/fluentassertions#3043 * Bump cspell from 8.19.3 to 9.0.0 by @dependabot in fluentassertions/fluentassertions#3045 * Bump cspell from 9.0.0 to 9.0.1 by @dependabot in fluentassertions/fluentassertions#3046 * Fix links to test suites in bug report template by @robvanuden in fluentassertions/fluentassertions#3047 **Full Changelog**: fluentassertions/fluentassertions@8.2.0...8.3.0 Commits viewable in [compare view](fluentassertions/fluentassertions@8.2.0...8.10.0). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Summary
This PR significantly improves the performance of
BeEquivalentTowhen comparing large unordered collections (usingWithoutStrictOrdering()). Focus benchmark: 500 complex objects vs 500 completely different objects.The work is split across five focused commits.
Changes
1. Replace O(n!) permutation search with O(n^2 log n) greedy assignment (Phase 3)
For n > 8 subjects, the original brute-force permutation search was replaced with a greedy strategy: compute all n^2 failure-count pairs, sort ascending, pick closest unassigned pair. n <= 8 keeps the exact search.
Also extracts
ReferentialComparerinto its own file and updates the benchmark target.2. Pre-cache collection index strings 0-1023
AsCollectionItem<T>(int index)calledindex.ToString()on every invocation. A static string[] cache for indices 0-1023 eliminates millions of small string allocations.3. Skip failure message formatting in dry-run comparisons
Phases 1 and 2 only need to count failures, not format them. Three-part change:
AssertionScope: newUseDryRunflag,RegisterFailure()counter,GetFailureCount()methodAssertionChain: whenUseDryRunis set, callRegisterFailure()instead of formattingLooselyOrderedEquivalencyStrategy: newTryToMatchCount()for counting; split caches (countCachefor ints,fullFailuresCachefor strings);FindClosestMismatchestakes separate count/full-failures delegatesEliminates ~3.25M FailureMessageFormatter calls and ~6.5M Regex.Replace calls from Phases 1+2.
4. Clone CyclicReferenceDetector per dry-run comparison
Dry-runs shared the parent context's
CyclicReferenceDetector. When the same object appears in multiple positions, the detector incorrectly flagged subsequent dry-runs as cyclic -- causing wrong counts and stack overflows. Fix: clone the detector for each dry-run call. Exposes the setter asinternal.5. Include nested scope failures in dry-run failure count
GetFailureCount()was missing failures from nested assertion scopes (they go intoassertionStrategy.FailureMessagesnot the local counter). Fix: returnfailureCount + assertionStrategy.FailureMessages.Count(). Also callscope.Discard()after counting to prevent leaking into the outer scope.Combined effect
Microbenchmark
BeEquivalentToWithDeeplyNestedStructuresBefore
After
PerformanceSpecs /
Compare_complex_collectionBefore
37-39 seconds
After
12-13 seconds