Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); pool async state in SslStream by wfurt · Pull Request #69418 · dotnet/runtime · GitHub
Skip to content

pool async state in SslStream - #69418

Merged
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool
May 26, 2022
Merged

pool async state in SslStream#69418
wfurt merged 4 commits into
dotnet:mainfrom
wfurt:asyncPool

Conversation

@wfurt

Copy link
Copy Markdown
Member

fixes#68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

@wfurt
wfurt requested review from a team and stephentoubMay 17, 2022 04:38
@wfurtwfurt self-assigned this May 17, 2022
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

fixes #68467

I had simple test code like

using(varsslClient=newSslStream(_client,leaveInnerStreamOpen:true,delegate{returntrue;}))using(varsslServer=newSslStream(_server,leaveInnerStreamOpen:true,delegate{returntrue;})){awaitTask.WhenAll(sslClient.AuthenticateAsClientAsync(clientOptions),sslServer.AuthenticateAsServerAsync(serverOptions));for(intj=0;j<100;j++){vart=sslClient.ReadAsync(_clientBuffer,CancellationToken.None);awaitsslServer.WriteAsync(_serverBuffer,CancellationToken.None);awaitt;}}

before the change it would allocate

image

after the change

image

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

Author:wfurt
Assignees:wfurt
Labels:

area-System.Net.Security

Milestone:-

@stephentoub

Copy link
Copy Markdown
Member

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect. Pooling the read method objects seems fairly reasonable, as in general we expect the SslStreams themselves to be relatively long-lived, pooled themselves as part of a connection pool. One of the difficulties with object pooling is that it can create artificial references from gen2 to gen0, and so if the state you're using is itself likely to be gen2, this negative aspect of pooling mostly goes away. With Connection: close, though, that suggests this is a scenario where these streams aren't being pooled and are much less likely to be gen2, and as such it's more likely that pooling the state machines will end up creating gen2 to gen0 references.

@wfurt

Copy link
Copy Markdown
MemberAuthor

EnsureFullTlsFrameAsync is also used in handshake so it should help little with Connection: close as well.

If you want to use it here, go ahead. But FWIW I think this case is a little more suspect.

The EnsureFullTlsFrameAsync as also called from Read so it is on hot path. I did not touch methods that are only in handshake e.g. called once per session. There are still easier wins for handshake.

@rzikm

Copy link
Copy Markdown
Member

The changeset now seems to include #69527, was this intentional?

@wfurt

Copy link
Copy Markdown
MemberAuthor

no, it was not intentional @rzikm.

@davidfowl

Copy link
Copy Markdown
Member

Nice! Are the other allocations the handshake?

@wfurt

Copy link
Copy Markdown
MemberAuthor

There are some @davidfowl. Some are easy to avoid some of them are more tricky.

It would be great if you can share any real workload you care about. With that we can investigate allocations as well as other runtime parts.

@rzikmrzikm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wfurt

Copy link
Copy Markdown
MemberAuthor

Failing Quic tests are unrelated and will be fixed by #69709 (msquic upgrade)

@wfurt
wfurt merged commit 008f128 into dotnet:mainMay 26, 2022
@wfurt
wfurt deleted the asyncPool branch May 26, 2022 13:28
@ghostghost locked as resolved and limited conversation to collaborators Jun 25, 2022
@karelzkarelz added this to the 7.0.0 milestone Jul 19, 2022
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add PoolingAsyncValueTaskMethodBuilder to System.Net.Security.SslStream.ReadAsyncInternal()

5 participants

@wfurt@stephentoub@rzikm@davidfowl@karelz