Skip to content

Fix expensive typo in JsonNode - #78130

Merged
eiriktsarpalis merged 1 commit into
dotnet:mainfrom
stephentoub:fixstaticoptions
Nov 10, 2022
Merged

Fix expensive typo in JsonNode#78130
eiriktsarpalis merged 1 commit into
dotnet:mainfrom
stephentoub:fixstaticoptions

Conversation

@stephentoub

Copy link
Copy Markdown
Member

Fixes#78089 (but assuming this is the right fix, we should backport this to release/7.0)

@ghost

ghost commented Nov 9, 2022

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #78089 (but assuming this is the right fix, we should backport this to release/7.0)

Author:stephentoub
Assignees:-
Labels:

area-System.Text.Json

Milestone:-

@stephentoub

Copy link
Copy Markdown
MemberAuthor

Using the repro benchmark from the linked issue:

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem.Text.Json.Nodes;[MemoryDiagnoser]publicpartialclassProgram{staticvoidMain(string[]args)=>BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);privateJsonObject[]_jsonObjs=newJsonObject[100_000];[Benchmark]publicvoidSum(){for(inti=0;i<100_000;i++){_jsonObjs[i]=newJsonObject(){["aaa"]=1,["bbbb"]=2,["cccc"]=newJsonArray(){newJsonObject(){["aaaa"]="aaaaa",["ffffff"]="ffffffffffffff"}}};}}}
MethodRuntimeMeanErrorStdDevRatioAllocated
Sum.NET 6.046.32 ms0.926 ms1.067 ms1.10127.41 MB
Sum.NET 7.0292.87 ms5.241 ms7.174 ms6.89313.13 MB
Sumpr42.42 ms0.617 ms0.547 ms1.00128.94 MB

@teo-tsirpanisteo-tsirpanis added this to the 8.0.0 milestone Nov 9, 2022
@EgorBo

Copy link
Copy Markdown
Member

I assume this scenario is not covered in the dotnet/performance suite or we somehow missed this.

@eiriktsarpaliseiriktsarpalis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, fix is spot on. We should backport this.

@eiriktsarpalis

Copy link
Copy Markdown
Member

Test failures are unrelated, mergin away.

@eiriktsarpalis
eiriktsarpalis merged commit b00a4ee into dotnet:mainNov 10, 2022
@eiriktsarpalis

Copy link
Copy Markdown
Member

/backport to release/7.0

@github-actions

Copy link
Copy Markdown
Contributor

Started backporting to release/7.0: https://github.com/dotnet/runtime/actions/runs/3435456807

@adamsitnik

Copy link
Copy Markdown
Member

@stephentoub we have added the "Allocation ratio" column to BDN based on your request: dotnet/BenchmarkDotNet#722 and I can see that you have removed it here while I would expect this particular issue to be perfect use case for it. May I ask why? Does the implementation do not meet your requirements?

@stephentoub

Copy link
Copy Markdown
MemberAuthor

I can see that you have removed it here while I would expect this particular issue to be perfect use case for it. May I ask why?

Because in this case I was focusing on the throughput impact rather than on the allocation impact, and removed extra columns to try to focus attention (e.g. RatioSD, Gen0, and Alloc Ratio). The 6x improvement in throughput was the primary thing to show, rather than the 2x reduction in allocation.

@stephentoub
stephentoub deleted the fixstaticoptions branch November 29, 2022 15:09
@dakersnar

Copy link
Copy Markdown
Contributor

I can confirm that the newly added unit tests (dotnet/performance#2714) are shown as heavily regressing in the .NET 7 vs 6 GA performance report.

Is there anything more that needs to be done to confirm we have addressed the problem with this backport, or are we confident this is a complete fix?

Details

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

ResultRatioAlloc DeltaOperating SystemBitProcessor NameModality
Slower0.31+12534Windows 11Arm64Microsoft SQ1 3.0 GHzbimodal
Slower0.43+12970macOS 13.0Arm64Apple M1
Slower0.34+12660Windows 10X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.26+12442Windows 10X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower0.30+12568Windows 10X64Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)several?
Slower0.38+12563Windows 11X64AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower0.33+12660ubuntu 18.04X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.28+7782raspbian 11ArmARMv7 Processor rev 3 (v7l)
Slower0.27+12513macOS Monterey 12.6X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

ResultRatioAlloc DeltaOperating SystemBitProcessor NameModality
Slower0.36+12469Windows 11Arm64Microsoft SQ1 3.0 GHzseveral?
Slower0.46+12995macOS 13.0Arm64Apple M1
Slower0.37+12691Windows 10X64Intel Xeon CPU E5-1650 v4 3.60GHzbimodal
Slower0.29+12533Windows 10X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower0.33+12578Windows 10X64Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower0.42+12552Windows 11X64AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower0.39+12652ubuntu 18.04X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.29+7724raspbian 11ArmARMv7 Processor rev 3 (v7l)
Slower0.30+12551macOS Monterey 12.6X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

ResultRatioAlloc DeltaOperating SystemBitProcessor NameModality
Slower0.43+2891666Windows 11Arm64Microsoft SQ1 3.0 GHzbimodal
Slower0.56+2887897macOS 13.0Arm64Apple M1
Slower0.49+2913026Windows 10X64Intel Xeon CPU E5-1650 v4 3.60GHzbimodal
Slower0.45+2964348Windows 10X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower0.39+2765957Windows 10X64Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower0.58+2912668Windows 11X64AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower0.48+2937050ubuntu 18.04X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.55+1675483raspbian 11ArmARMv7 Processor rev 3 (v7l)several?
Slower0.42+2938707macOS Monterey 12.6X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

ResultRatioAlloc DeltaOperating SystemBitProcessor NameModality
Slower0.51+10563Windows 11Arm64Microsoft SQ1 3.0 GHz
Slower0.61+10136macOS 13.0Arm64Apple M1
Slower0.55+10666Windows 10X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.46+10542Windows 10X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower0.51+10581Windows 10X64Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower0.58+10596Windows 11X64AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower0.50+10711ubuntu 18.04X64Intel Xeon CPU E5-1650 v4 3.60GHz
Slower0.44+6202raspbian 11ArmARMv7 Processor rev 3 (v7l)
Slower0.45+10558macOS Monterey 12.6X64Intel Core i7-5557U CPU 3.10GHz (Broadwell)

(etc for all other benchmarks)

@eiriktsarpalis

Copy link
Copy Markdown
Member

Yes, we're fairly confident that the regression has been addressed. I've verified that the particular benchmark has parity between .NET 6 and the patched .NET 7 branch.

@ghostghost locked as resolved and limited conversation to collaborators Jan 2, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.net 7 cycles Performance slow

6 participants

@stephentoub@EgorBo@eiriktsarpalis@adamsitnik@dakersnar@teo-tsirpanis