Skip to content

Optimize index bounds check for immutable sorted set and list - #53266

Merged
stephentoub merged 4 commits into
dotnet:mainfrom
amd:immutableBounds
Jul 9, 2021
Merged

Optimize index bounds check for immutable sorted set and list#53266
stephentoub merged 4 commits into
dotnet:mainfrom
amd:immutableBounds

Conversation

@L2

@L2L2 commented May 26, 2021

Copy link
Copy Markdown
Contributor
  • The bounds check to determine if a given index is >= 0 and < this.Count
    is only necessary on the first call to ItemRef.

  • The recursive steps within ItemRef do not need to continuously
    do this bounds check on these immutable data structures.

  • Proof:
    Elimination of index >= 0 bounds check:
    The first call to ItemRef checks if index >= 0.
    If we recurse on the left node, the index value does not change.
    If we recurse on the right node, index > _left._count. Then
    index - _left._count - 1 >= 0.

    Elimination of index < this.Count:
    The first call to ItemRef checks if index < this.Count. Then
    the given index must lie somewhere in this tree and
    (**) index < this.Count == left.Count + right.Count + 1.
    If we recurse on the left node, the index value does not change
    and a check is already made to determine that index < _left.Count.
    If we recurse on the right node, then we need to be sure that
    index - _left.count - 1 < _right.Count. But this is just a
    rearrangement of (**).

- The bounds check to determine if a given index is >= 0 and < this.Count
is only necessary on the first call to ItemRef.
- The recursive steps within ItemRef do not need to continuously
do this bounds check on these immutable data structures.
- Proof:
Elimination of index >= 0 bounds check:
The first call to ItemRef checks if index >= 0.
If we recurse on the left node, the index value does not change.
If we recurse on the right node, index > _left._count. Then
index - _left._count - 1 >= 0.
Elimination of index < this.Count:
The first call to ItemRef checks if index < this.Count. Then
the given index must lie somewhere in this tree and
(**) index < this.Count == left.Count + right.Count + 1.
If we recurse on the left node, the index value does not change
and a check is already made to determine that index < _left.Count.
If we recurse on the right node, then we need to be sure that
index - _left.count - 1 < _right.Count. But this is just a
rearrangement of (**).
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

Issue Details
  • The bounds check to determine if a given index is >= 0 and < this.Count
    is only necessary on the first call to ItemRef.

  • The recursive steps within ItemRef do not need to continuously
    do this bounds check on these immutable data structures.

  • Proof:
    Elimination of index >= 0 bounds check:
    The first call to ItemRef checks if index >= 0.
    If we recurse on the left node, the index value does not change.
    If we recurse on the right node, index > _left._count. Then
    index - _left._count - 1 >= 0.

    Elimination of index < this.Count:
    The first call to ItemRef checks if index < this.Count. Then
    the given index must lie somewhere in this tree and
    (**) index < this.Count == left.Count + right.Count + 1.
    If we recurse on the left node, the index value does not change
    and a check is already made to determine that index < _left.Count.
    If we recurse on the right node, then we need to be sure that
    index - _left.count - 1 < _right.Count. But this is just a
    rearrangement of (**).

Author:L2
Assignees:-
Labels:

area-System.Collections

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

Thanks. Can you share benchmark results showing that the extra code required here to avoid that range check actually makes a meaningful difference? (Note that range check itself could be optimized slightly as well, to Requires.Range((uint)index < (uint)this.Count, nameof(index));, which would further reduce the impact of avoiding it.)

@L2

L2 commented May 26, 2021

Copy link
Copy Markdown
ContributorAuthor

Sure @stephentoub

dotnet/performance diff with/without this change using --filter "*ImmutableSortedSet*" "*ImmutableList*"

summary:
better: 6, geomean: 1.202
total diff: 6

No Slower results for the provided threshold = 2% and noise filter = 25ns.

Fasterbase/diffBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)1.406618.824735.50
System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512)1.336613.094984.01
System.Collections.IterateFor<Int32>.ImmutableList(Size: 512)1.276485.475117.11
System.Collections.IterateFor<String>.ImmutableList(Size: 512)1.095579.585116.40
System.Collections.IterateForEach<Int32>.ImmutableSortedSet(Size: 512)1.0912384.0411380.41
System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512)1.0815290.2514142.94

@L2

L2 commented May 26, 2021

Copy link
Copy Markdown
ContributorAuthor

@stephentoub thanks, I did encounter some places with these Requires.Range checks that could benefit from the optimization you listed. Should I place those changes in another PR or keep it in this one?

@stephentoub

Copy link
Copy Markdown
Member

dotnet/performance diff with/without this change

Thanks for sharing numbers. I was surprised by the magnitude of the differences you're seeing, so I tried locally, and I'm not seeing results anything like that, e.g.

dotnet run -c Release -f net6.0 --filter "*ImmutableSortedSet*" "*ImmutableList*" --corerun d:\coreclrtest\main\corerun.exe d:\coreclrtest\pr\corerun.exe --join
TypeMethodJobToolchainSizeMeanErrorStdDevMedianMinMaxRatioRatioSDGen 0Gen 1Gen 2Allocated
ContainsTrueComparerImmutableSortedSetJob-TGGIVO\main\corerun.exe51225.751 us0.0948 us0.0886 us25.745 us25.624 us25.917 us1.000.00----
ContainsTrueComparerImmutableSortedSetJob-EZHMIY\pr\corerun.exe51225.579 us0.0777 us0.0727 us25.586 us25.435 us25.706 us0.990.00----
ContainsTrueComparerImmutableSortedSetJob-TGGIVO\main\corerun.exe512249.051 us3.7624 us3.3353 us247.656 us246.321 us255.871 us1.000.00----
ContainsTrueComparerImmutableSortedSetJob-EZHMIY\pr\corerun.exe512253.804 us0.5012 us0.4443 us253.849 us252.679 us254.556 us1.020.01----
IterateForImmutableListJob-TGGIVO\main\corerun.exe51212.231 us0.0654 us0.0612 us12.229 us12.134 us12.352 us1.000.00----
IterateForImmutableListJob-EZHMIY\pr\corerun.exe51210.500 us0.0343 us0.0286 us10.499 us10.434 us10.552 us0.860.00----
IterateForImmutableListJob-TGGIVO\main\corerun.exe51211.932 us0.0684 us0.0640 us11.929 us11.826 us12.060 us1.000.00----
IterateForImmutableListJob-EZHMIY\pr\corerun.exe51210.661 us0.0537 us0.0476 us10.668 us10.559 us10.753 us0.890.01----
IterateForImmutableSortedSetJob-TGGIVO\main\corerun.exe51212.075 us0.0432 us0.0404 us12.067 us12.009 us12.159 us1.000.00----
IterateForImmutableSortedSetJob-EZHMIY\pr\corerun.exe51210.912 us0.0579 us0.0541 us10.909 us10.831 us11.017 us0.900.00----
IterateForImmutableSortedSetJob-TGGIVO\main\corerun.exe51212.346 us0.0657 us0.0614 us12.355 us12.243 us12.440 us1.000.00----
IterateForImmutableSortedSetJob-EZHMIY\pr\corerun.exe51212.396 us0.0393 us0.0368 us12.394 us12.333 us12.448 us1.000.00----
ContainsFalseImmutableListJob-TGGIVO\main\corerun.exe5121,138.721 us10.7122 us9.4961 us1,140.382 us1,126.251 us1,155.342 us1.000.00---1 B
ContainsFalseImmutableListJob-EZHMIY\pr\corerun.exe5121,119.929 us13.2908 us12.4322 us1,119.789 us1,101.062 us1,145.184 us0.980.01---1 B
ContainsFalseImmutableListJob-TGGIVO\main\corerun.exe5122,465.770 us17.6445 us15.6414 us2,466.149 us2,445.180 us2,494.746 us1.000.00---1 B
ContainsFalseImmutableListJob-EZHMIY\pr\corerun.exe5122,563.138 us13.3149 us12.4548 us2,564.662 us2,543.398 us2,588.892 us1.040.01---1 B
ContainsTrueImmutableListJob-TGGIVO\main\corerun.exe512578.459 us11.2031 us11.5048 us572.648 us568.086 us606.604 us1.000.00----
ContainsTrueImmutableListJob-EZHMIY\pr\corerun.exe512537.503 us10.3608 us8.6518 us534.372 us529.038 us559.094 us0.930.02----
ContainsTrueImmutableListJob-TGGIVO\main\corerun.exe5121,241.312 us6.5191 us5.4437 us1,241.169 us1,233.714 us1,251.366 us1.000.00---1 B
ContainsTrueImmutableListJob-EZHMIY\pr\corerun.exe5121,246.525 us12.9054 us12.0717 us1,243.906 us1,233.100 us1,269.617 us1.000.01---1 B
ContainsFalseImmutableSortedSetJob-TGGIVO\main\corerun.exe51230.266 us0.0997 us0.0884 us30.243 us30.170 us30.499 us1.000.00----
ContainsFalseImmutableSortedSetJob-EZHMIY\pr\corerun.exe51230.336 us0.0898 us0.0749 us30.328 us30.198 us30.494 us1.000.00----
ContainsFalseImmutableSortedSetJob-TGGIVO\main\corerun.exe512255.948 us0.7200 us0.6383 us255.751 us255.221 us257.445 us1.000.00----
ContainsFalseImmutableSortedSetJob-EZHMIY\pr\corerun.exe512265.701 us0.9717 us0.7586 us265.424 us264.676 us267.011 us1.040.00----
ContainsTrueImmutableSortedSetJob-TGGIVO\main\corerun.exe51227.005 us0.2554 us0.2389 us26.969 us26.734 us27.488 us1.000.00----
ContainsTrueImmutableSortedSetJob-EZHMIY\pr\corerun.exe51227.067 us0.0620 us0.0580 us27.051 us26.998 us27.177 us1.000.01----
ContainsTrueImmutableSortedSetJob-TGGIVO\main\corerun.exe512194.003 us0.6218 us0.5193 us193.893 us193.240 us195.092 us1.000.00----
ContainsTrueImmutableSortedSetJob-EZHMIY\pr\corerun.exe512204.322 us2.8966 us2.7095 us204.841 us200.911 us208.696 us1.050.01----
CtorFromCollectionImmutableListJob-TGGIVO\main\corerun.exe5128.542 us0.0679 us0.0602 us8.524 us8.464 us8.685 us1.000.003.89990.4409-24,624 B
CtorFromCollectionImmutableListJob-EZHMIY\pr\corerun.exe5128.793 us0.1887 us0.2173 us8.809 us8.478 us9.129 us1.040.033.89130.4437-24,624 B
CtorFromCollectionImmutableListJob-TGGIVO\main\corerun.exe51211.678 us0.0643 us0.0601 us11.667 us11.562 us11.788 us1.000.003.88600.4164-24,624 B
CtorFromCollectionImmutableListJob-EZHMIY\pr\corerun.exe51211.707 us0.1028 us0.0961 us11.673 us11.590 us11.930 us1.000.013.88030.4157-24,624 B
CtorFromCollectionImmutableSortedSetJob-TGGIVO\main\corerun.exe51215.941 us0.1666 us0.1558 us15.903 us15.678 us16.239 us1.000.004.21700.5035-26,736 B
CtorFromCollectionImmutableSortedSetJob-EZHMIY\pr\corerun.exe51216.239 us0.2711 us0.2536 us16.313 us15.779 us16.568 us1.020.024.21420.5187-26,736 B
CtorFromCollectionImmutableSortedSetJob-TGGIVO\main\corerun.exe512282.278 us0.7908 us0.7010 us282.259 us281.298 us283.706 us1.000.004.4643--28,784 B
CtorFromCollectionImmutableSortedSetJob-EZHMIY\pr\corerun.exe512286.381 us0.7881 us0.6987 us286.041 us285.556 us287.849 us1.010.004.5455--28,784 B
IterateForEachImmutableListJob-TGGIVO\main\corerun.exe51211.316 us0.0252 us0.0236 us11.325 us11.277 us11.348 us1.000.00----
IterateForEachImmutableListJob-EZHMIY\pr\corerun.exe51212.338 us0.0422 us0.0374 us12.330 us12.262 us12.412 us1.090.00----
IterateForEachImmutableListJob-TGGIVO\main\corerun.exe51224.239 us0.2083 us0.1846 us24.242 us24.000 us24.628 us1.000.00----
IterateForEachImmutableListJob-EZHMIY\pr\corerun.exe51223.989 us0.1145 us0.0956 us24.007 us23.698 us24.078 us0.990.01----
IterateForEachImmutableSortedSetJob-TGGIVO\main\corerun.exe51212.120 us0.0687 us0.0609 us12.115 us12.028 us12.257 us1.000.00----
IterateForEachImmutableSortedSetJob-EZHMIY\pr\corerun.exe51211.638 us0.0576 us0.0511 us11.641 us11.533 us11.718 us0.960.01----
IterateForEachImmutableSortedSetJob-TGGIVO\main\corerun.exe51223.186 us0.0741 us0.0693 us23.194 us23.048 us23.308 us1.000.00----
IterateForEachImmutableSortedSetJob-EZHMIY\pr\corerun.exe51223.982 us0.1367 us0.1212 us23.969 us23.718 us24.193 us1.030.01----

Can you share more about how you're getting your numbers?

@stephentoub

Copy link
Copy Markdown
Member

The code changes look fine, but for the most part this doesn't appear to consistently move the needle, so I'm not sure it's worth the churn.

@L2

L2 commented May 28, 2021

Copy link
Copy Markdown
ContributorAuthor

Thanks @stephentoub , it looks like some recent commits have changed the perf numbers. Here's a breakdown of my investigation (Windows x64):

without my change + base commit = b443b8c (May 24, 2021) (The numbers I provided earlier were based on this)
with my change + base commit = b443b8c (May 24, 2021) (The numbers I provided earlier were based on this)
without my change + base commit = d43e886 (May 28, 2021)
with my change + base commit = d43e886 (May 28, 2021)

Today I cloned dotnet/runtime and created the four coreruns above and reran the microbenchmarks:

Here's an example of the commands I use:

py .\scripts\benchmarks_ci.py -c Release -f net6.0 --bdn-artifacts "C:\W052821\results\with_change" --corerun "C:\W052821\with_change\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\corerun.exe" --filter "*ImmutableSortedSet*" "*ImmutableList*"
py .\scripts\benchmarks_ci.py -c Release -f net6.0 --bdn-artifacts "C:\W052821\results\without_change" --corerun "C:\W052821\without_change\testhost\net6.0-windows-Release-x64\shared\Microsoft.NETCore.App\6.0.0\corerun.exe" --filter "*ImmutableSortedSet*" "*ImmutableList*"

then I diff the results using dotnet/performance's resultsComparer:

dotnet run --base "C:\W052821\results\without_change\" --diff "C:\W052821\results\with_change\"--threshold 2% --noise 25ns --full-id

Results:

without my change + base commit = b443b8c (May 24, 2021)
with my change + base commit = b443b8c (May 24, 2021) [DIFF]

summary:
better: 7, geomean: 1.191
worse: 1, geomean: 1.031
total diff: 8

Slowerdiff/baseBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateForEach.ImmutableSortedSet(Size: 512)1.0322179.7022860.75
Fasterbase/diffBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.416615.044681.89
System.Collections.IterateFor.ImmutableList(Size: 512)1.346480.184850.76
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.236629.845384.70
System.Collections.IterateFor.ImmutableList(Size: 512)1.175595.224800.09
System.Collections.IterateForEach.ImmutableSortedSet(Size: 512)1.0912401.4011399.09
System.Collections.ContainsTrue.ImmutableSortedSet(Size: 512)1.0915318.1914107.30
System.Collections.ContainsTrueComparer.ImmutableSortedSet(Size: 512)1.06212987.67200160.62

without my change + base commit = d43e886 (May 28, 2021)
with my change + base commit = d43e886 (May 28, 2021) [DIFF]

summary:
better: 6, geomean: 1.101
worse: 1, geomean: 1.083
total diff: 7

Slowerdiff/baseBase Median (ns)Diff Median (ns)Modality
System.Collections.ContainsTrue.ImmutableSortedSet(Size: 512)1.0814120.7915288.74several?
Fasterbase/diffBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor.ImmutableList(Size: 512)1.256497.435215.18
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.155587.324839.56
System.Collections.IterateForEach.ImmutableSortedSet(Size: 512)1.0812242.4811376.06
System.Collections.IterateFor.ImmutableList(Size: 512)1.075592.335229.02several?
System.Collections.CtorFromCollection.ImmutableSortedSet(Size: 512)1.05273723.27261787.71
System.Collections.IterateForEach.ImmutableList(Size: 512)1.0322647.7821959.21

Notes:
Rerunning on a fresh clone and build with base = b443b8c (May 24, 2021) with/without my change show the numbers being consistent on my machine with the numbers I provided earlier.

Building with the new base = d43e886 (May 28, 2021) seems to have changed some of the perf numbers around. Some faster and some slower than before:

without my change + base commit = b443b8c (May 24, 2021)
without my change + base commit = d43e886 (May 28, 2021) [DIFF]

summary:
better: 5, geomean: 1.138
worse: 3, geomean: 1.100
total diff: 8

Slowerdiff/baseBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor.ImmutableList(Size: 512)1.165595.226497.43
System.Collections.CtorFromCollection.ImmutableSortedSet(Size: 512)1.08252765.37273723.27
System.Collections.IterateForEach.ImmutableList(Size: 512)1.0621371.0622647.78
Fasterbase/diffBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.216615.045447.31
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.196629.845587.32
System.Collections.IterateFor.ImmutableList(Size: 512)1.166480.185592.33several?
System.Collections.ContainsTrue.ImmutableSortedSet(Size: 512)1.0815318.1914120.79several?
System.Collections.ContainsTrueComparer.ImmutableSortedSet(Size: 512)1.06212987.67201713.26

So this change still shows some decent results with the latest base commit = d43e886 (May 28, 2021), but you're right not nearly as good as the numbers on base commit base commit = b443b8c (May 24, 2021). I'll follow your recommendation.

Change from internal to private for unchecked methods.
Co-authored-by: Stephen Toub <stoub@microsoft.com>
@L2

L2 commented Jun 9, 2021

Copy link
Copy Markdown
ContributorAuthor

Latest .NET6 (commit ac87f00) before/after this change:

summary:
better: 5, geomean: 1.160
worse: 2, geomean: 1.053
total diff: 7

Slowerdiff/baseBase Median (ns)Diff Median (ns)Modality
System.Collections.ContainsTrue.ImmutableSortedSet(Size: 512)1.0814111.2815277.27
System.Collections.IterateForEach.ImmutableSortedSet(Size: 512)1.0222045.8522582.00
Fasterbase/diffBase Median (ns)Diff Median (ns)Modality
System.Collections.IterateFor.ImmutableList(Size: 512)1.246476.435206.49
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.236476.555246.25
System.Collections.IterateFor.ImmutableSortedSet(Size: 512)1.165593.194840.51several?
System.Collections.IterateFor.ImmutableList(Size: 512)1.165595.044842.67
System.Collections.IterateForEach.ImmutableSortedSet(Size: 512)1.0211791.1611525.10

@stephentoubstephentoub left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@stephentoub
stephentoub merged commit 8b42b7f into dotnet:mainJul 9, 2021
@L2
L2 deleted the immutableBounds branch July 14, 2021 20:24
@ghostghost locked as resolved and limited conversation to collaborators Aug 13, 2021
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@L2@stephentoub