[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553

Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from
MihaZupan:backport-json-net7
Feb 9, 2023
Merged

[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from
MihaZupan:backport-json-net7

Conversation

@MihaZupan

@MihaZupanMihaZupan commented Jan 12, 2023•
edited
Loading

Copy link
Copy Markdown
Member

Backport of a minimized change of #79386 to release/7.0

Customer Impact

HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).

There are three main ways to get the response content from HttpClient:

  1. Using the one-line helper methods 
    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content 
    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call 
    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design

This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).
This allows us to present consistent HttpClient behavior across the board.

Testing

I added targeted CI tests that confirm limits are consistently enforced.

Risk

The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.
The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.

The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.

@MihaZupanMihaZupan added this to the 7.0.x milestone Jan 12, 2023
@MihaZupan
MihaZupan requested a review from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan self-assigned this Jan 12, 2023
@ghost

ghost commented Jan 12, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Backport of a minimized change of #79386 to release/7.0

Customer Impact

TODO

Testing

TODO

Risk

TODO

Author:MihaZupan
Assignees:MihaZupan
Labels:

area-System.Net.Http

Milestone:7.0.x

@build-analysisbuild-analysisBot mentioned this pull request Jan 12, 2023
Closed
@carlossanlop

carlossanlop commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?

Also, there are networking-related CI failures. Can you please investigate them?

@MihaZupan

MihaZupan commented Jan 13, 2023

Copy link
Copy Markdown
MemberAuthor

Build failure is known according to build analysis.

@carlossanlopcarlossanlop added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Jan 13, 2023
@carlossanlop

carlossanlop commented Jan 13, 2023

Copy link
Copy Markdown
Contributor

Talked to @MihaZupan. This will go in next month.

ManickaP
ManickaP approved these changes Jan 18, 2023
View reviewed changes

@ManickaPManickaP left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@MihaZupanMihaZupan added NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) Servicing-consider Issue for next servicing release review and removed NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) labels Feb 6, 2023
@karelz

karelz commented Feb 9, 2023

Copy link
Copy Markdown
Member

Approved by Tactics via email by @SteveMCarroll on 2/9.
Adding Servicing-approved label to the PR.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Feb 9, 2023
@carlossanlopcarlossanlop modified the milestones: 7.0.x, 7.0.4Feb 9, 2023
@carlossanlop

carlossanlop commented Feb 9, 2023

Copy link
Copy Markdown
Contributor

Approved by Tactics for 7.0.4.
Signed-off by area owners.
CI failure was a timeout cancellation in staging.
Required OOB package changes look good in System.Net.Http.Json.
Ready to merge. :shipit:

@carlossanlop
carlossanlop merged commit 3c73365 into dotnet:release/7.0Feb 9, 2023
@carlossanlop

carlossanlop commented Feb 9, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:

https://github.com/dotnet/runtime/runs/11233952875

Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.

Edit: I opened #81914 and pinged Viktor/Eric for help.

@carlossanlopcarlossanlop mentioned this pull request Feb 9, 2023
Closed
@ghostghost locked as resolved and limited conversation to collaborators Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects

None yet

Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync - #80553
Merged
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Feb 9, 2023
Merged
[release/7.0] Enforce HttpClient limits on GetFromJsonAsync#80553
carlossanlop merged 2 commits into
dotnet:release/7.0from 
MihaZupan:backport-json-net7
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/7.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This change changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 7.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/7.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:7.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Tomorrow is the last day for merging backports for the February Release. Can you please fill out the template, making sure the customer impact is clearly described, add the servicing-consider label, and send an email to Tactics requesting approval?
Also, there are networking-related CI failures. Can you please investigate them?
@MihaZupan
MihaZupan
commented
Jan 13, 2023

Copy link


Copy Markdown

MemberAuthor
Build failure is known according to build analysis.
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
added

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 6, 2023
@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
7.0.x, 7.0.4Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 7.0.4.

Signed-off by area owners.

CI failure was a timeout cancellation in staging.

Required OOB package changes look good in System.Net.Http.Json.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 3c73365
into
dotnet:release/7.0Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
@MihaZupan@ManickaP I'm looking at the rolling builds for the release/7.0 branch, and after merging this PR, I am seeing a nuget issue related to System.Net.Http.Json:
https://github.com/dotnet/runtime/runs/11233952875
Can you please take a look? I'll open an issue to track this. It needs to get fixed before Monday EOD. That's the day we close the servicing branches.
Edit: I opened #81914 and pinged Viktor/Eric for help.
@carlossanlopcarlossanlop
mentioned this pull request
Feb 9, 2023

Closed

@ghostghost
locked as resolved and limited conversation to collaborators
Mar 12, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

7.0.4

Development

Successfully merging this pull request may close these issues.

4 participants

@MihaZupan@carlossanlop@karelz@ManickaP