Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Improve perf of CredentialCache.GetCredential by rzikm · Pull Request #103714 · dotnet/runtime · GitHub
Skip to content

Improve perf of CredentialCache.GetCredential - #103714

Merged
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf
Jun 25, 2024
Merged

Improve perf of CredentialCache.GetCredential#103714
rzikm merged 2 commits into
dotnet:mainfrom
rzikm:credscache-perf

Conversation

@rzikm

Copy link
Copy Markdown
Member

Fixes#101991.
Fixes#102281.

Not sure what introduced the original regression, but we can save some time by first checking if candidate credential can have longer prefix match before we start comparing auth schemes and uri, and we can stop early if we find exact match.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@rzikm
rzikm marked this pull request as ready for review June 19, 2024 17:18
@rzikm
rzikm requested a review from a teamJune 19, 2024 17:39
@rzikm

rzikm commented Jun 19, 2024

Copy link
Copy Markdown
MemberAuthor

/benchmark micro aspnet-perf-lin libs --variable filter=*GetCredential_Uri

@MihaZupan

Copy link
Copy Markdown
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=new(){{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://notfound",10)][Arguments("http://name5",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString());cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
Job-OMNTDN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-SHAJJN : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://name510395.20 ns0.447 ns1.00
GetCredential_UriPRhttp://name510330.18 ns0.224 ns0.84
GetCredential_UriMainhttp://notfound075.30 ns0.073 ns1.00
GetCredential_UriPRhttp://notfound075.17 ns0.091 ns1.00
GetCredential_UriMainhttp://notfound10240.26 ns0.968 ns1.00
GetCredential_UriPRhttp://notfound10304.42 ns0.310 ns1.27

BDN_Artifacts.zip

@rzikm

rzikm commented Jun 24, 2024

Copy link
Copy Markdown
MemberAuthor

I took a look at why we regressed the one scenario, and it seems that that particular benchmark is measuring a corner case. I locally editted the benchmarks so that the creds in the cache on the same host, but different prefix (http://name/prefix{i}/) and added couple more test cases and the changes actually improve common scenario. Numbers below are this PR compared to main before introducing the regression.

BenchmarkDotNet v0.13.13-nightly.20240311.145, Windows 11 (10.0.22631.3737/23H2/2023Update/SunValley3)
Intel Core i9-10900K CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK 9.0.100-preview.5.24307.3
[Host] : .NET 9.0.0 (9.0.24.30607), X64 RyuJIT AVX2
Job-WTZYSK : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
Job-CDIHKU : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
PowerPlanMode=00000000-0000-0000-0000-000000000000 IterationTime=250ms MaxIterationCount=20
MinIterationCount=15 WarmupCount=1
| Method | Job | Toolchain | uriString | uriCount | Mean | Error | StdDev | Median | Min | Max | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|------------------ |----------- |------------------- |--------------------------------- |--------- |----------:|---------:|---------:|----------:|----------:|----------:|------:|--------:|-------:|----------:|------------:|
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://differentHost | 10 | 273.23 ns | 4.014 ns | 3.755 ns | 273.96 ns | 265.50 ns | 276.82 ns | 1.21 | 0.02 | 0.0248 | 264 B | 1.43 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://differentHost | 10 | 225.99 ns | 3.684 ns | 3.266 ns | 225.76 ns | 221.59 ns | 232.48 ns | 1.00 | 0.02 | 0.0171 | 184 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/diff/path | 10 | 317.86 ns | 5.780 ns | 5.124 ns | 317.80 ns | 308.61 ns | 326.92 ns | 0.90 | 0.02 | 0.0259 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/diff/path | 10 | 352.02 ns | 6.018 ns | 5.629 ns | 349.61 ns | 346.69 ns | 365.42 ns | 1.00 | 0.02 | 0.0258 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/differentPrefix/path | 10 | 456.69 ns | 8.940 ns | 8.363 ns | 454.95 ns | 445.83 ns | 470.55 ns | 0.99 | 0.02 | 0.0270 | 288 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/differentPrefix/path | 10 | 459.11 ns | 7.361 ns | 6.885 ns | 457.80 ns | 448.95 ns | 474.82 ns | 1.00 | 0.02 | 0.0263 | 288 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://name/prefix5/path | 10 | 381.24 ns | 7.080 ns | 6.623 ns | 380.00 ns | 372.30 ns | 392.75 ns | 0.81 | 0.02 | 0.0258 | 272 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://name/prefix5/path | 10 | 469.81 ns | 5.792 ns | 5.418 ns | 469.72 ns | 460.92 ns | 478.05 ns | 1.00 | 0.02 | 0.0246 | 272 B | 1.00 |
| | | | | | | | | | | | | | | | |
| GetCredential_Uri | Job-WTZYSK | \9.0.0\corerun.exe | http://notfound | 0 | 59.25 ns | 1.229 ns | 1.150 ns | 58.77 ns | 57.81 ns | 61.34 ns | 0.98 | 0.02 | 0.0051 | 56 B | 1.00 |
| GetCredential_Uri | Job-CDIHKU | \main\corerun.exe | http://notfound | 0 | 60.65 ns | 0.773 ns | 0.723 ns | 60.47 ns | 59.68 ns | 62.08 ns | 1.00 | 0.02 | 0.0052 | 56 B | 1.00 |

Since the regressed scenario is when we are querying a completely different host, I think the regression is not a big problem.

@dotnetdotnet deleted a comment from EgorBotJun 24, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

@EgorBot

// Licensed to the .NET Foundation under one or more agreements.// The .NET Foundation licenses this file to you under the MIT license.// See the LICENSE file in the project root for more information.usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Collections.Generic;usingSystem.Net;publicclassCredentialCacheTests{privateconststringUriPrefix="http://name/prefix";privateconststringHostPrefix="name";privateconstintPort=80;privateconststringAuthenticationType="authType";privatestaticreadonlyNetworkCredentials_credential=newNetworkCredential();privatereadonlyDictionary<(inturiCount,inthostPortCount),CredentialCache>_caches=newDictionary<(inturiCount,inthostPortCount),CredentialCache>{{(0,0),CreateCredentialCache(0,0)},{(0,10),CreateCredentialCache(0,10)},{(10,0),CreateCredentialCache(10,0)},{(10,10),CreateCredentialCache(10,10)}};[Benchmark][Arguments("http://notfound",0)][Arguments("http://differentHost",10)][Arguments("http://name/prefix5/path",10)][Arguments("http://name/differentPrefix/path",10)][Arguments("http://name/diff/path",10)]publicNetworkCredentialGetCredential_Uri(stringuriString,inturiCount){CredentialCachecc=_caches[(uriCount:uriCount,hostPortCount:0)];returncc.GetCredential(newUri(uriString),AuthenticationType);}privatestaticCredentialCacheCreateCredentialCache(inturiCount,inthostPortCount){varcc=newCredentialCache();for(inti=0;i<uriCount;i++){Uriuri=newUri(UriPrefix+i.ToString()+"/");cc.Add(uri,AuthenticationType,s_credential);}for(inti=0;i<hostPortCount;i++){stringhost=HostPrefix+i.ToString();cc.Add(host,Port,AuthenticationType,s_credential);}returncc;}}

@EgorBot

Copy link
Copy Markdown
Benchmark results on Intel
BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores
Job-PJPJZA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job-YATEEO : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodToolchainuriStringuriCountMeanErrorRatio
GetCredential_UriMainhttp://differentHost10270.15 ns0.622 ns1.00
GetCredential_UriPRhttp://differentHost10342.75 ns1.019 ns1.27
GetCredential_UriMainhttp:(...)/path [21]10500.69 ns0.553 ns1.00
GetCredential_UriPRhttp:(...)/path [21]10375.33 ns0.780 ns0.75
GetCredential_UriMainhttp:(...)/path [32]10667.06 ns0.513 ns1.00
GetCredential_UriPRhttp:(...)/path [32]10589.57 ns2.944 ns0.88
GetCredential_UriMainhttp:(...)/path [24]10659.35 ns0.849 ns1.00
GetCredential_UriPRhttp:(...)/path [24]10456.99 ns0.295 ns0.69
GetCredential_UriMainhttp://notfound077.91 ns0.176 ns1.00
GetCredential_UriPRhttp://notfound076.56 ns0.173 ns0.98

BDN_Artifacts.zip

@rzikm

Copy link
Copy Markdown
MemberAuthor

@dotnet/ncl Can I get a review?

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This can still be noise compared to actual TLS.

Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Perf] Linux/arm64: 2 Regressions on 4/30/2024 1:50:24 PM [Perf] Linux/x64: 1 Regression on 4/30/2024 10:57:18 AM

5 participants

@rzikm@MihaZupan@EgorBot@wfurt@karelz