[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552

Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from
MihaZupan:backport-json-net6
Feb 9, 2023
Merged

[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from
MihaZupan:backport-json-net6

Conversation

@MihaZupan

@MihaZupanMihaZupan commented Jan 12, 2023•
edited by karelz
Loading

Copy link
Copy Markdown
Member

Backport of a minimized change of #79386 to release/6.0

Customer Impact

HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).

There are three main ways to get the response content from HttpClient:

  1. Using the one-line helper methods 
    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content 
    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call 
    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design

This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).
This allows us to present consistent HttpClient behavior across the board.

Testing

I added targeted CI tests that confirm limits are consistently enforced.

Risk

The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.
The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.

The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.

@MihaZupanMihaZupan added this to the 6.0.x milestone Jan 12, 2023
@MihaZupan
MihaZupan requested a review from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan self-assigned this Jan 12, 2023
@ghost

ghost commented Jan 12, 2023

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Backport of a minimized change of #79386 to release/6.0

Customer Impact

TODO

Testing

TODO

Risk

TODO

Author:MihaZupan
Assignees:MihaZupan
Labels:

area-System.Net.Http

Milestone:6.0.x

@build-analysisbuild-analysisBot mentioned this pull request Jan 12, 2023
Closed
@carlossanlop

carlossanlop commented Jan 12, 2023

Copy link
Copy Markdown
Contributor

Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.

@MihaZupanMihaZupan reopened this Jan 13, 2023
@carlossanlopcarlossanlop added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Jan 13, 2023
@carlossanlop

carlossanlop commented Jan 13, 2023

Copy link
Copy Markdown
Contributor

Talked to @MihaZupan. This will go in next month.

ManickaP
ManickaP approved these changes Jan 18, 2023
View reviewed changes

@ManickaPManickaP left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@MihaZupanMihaZupan reopened this Jan 31, 2023
@MihaZupanMihaZupan mentioned this pull request Feb 6, 2023
Closed
@MihaZupan

MihaZupan commented Feb 6, 2023

Copy link
Copy Markdown
MemberAuthor

Failures are #81391, #81544.

@MihaZupanMihaZupan added Servicing-consider Issue for next servicing release review and removed NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) labels Feb 8, 2023
@carlossanlop

carlossanlop commented Feb 8, 2023

Copy link
Copy Markdown
Contributor

Hey @ViktorHofer what should we do about this failure?:

.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer

ViktorHofer commented Feb 8, 2023

Copy link
Copy Markdown
Member

That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.

We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.

You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here:

runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5 in bd54093

</PropertyGroup>

@carlossanlopcarlossanlop mentioned this pull request Feb 8, 2023
Merged
@karelz

karelz commented Feb 9, 2023

Copy link
Copy Markdown
Member

Approved by Tactics via email by @SteveMCarroll on 2/9.
Adding Servicing-approved label to the PR.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Feb 9, 2023
@carlossanlopcarlossanlop modified the milestones: 6.0.x, 6.0.15Feb 9, 2023
@carlossanlop

carlossanlop commented Feb 9, 2023

Copy link
Copy Markdown
Contributor

Approved by Tactics for 6.0.15.
Signed-off by area owners.
CI failures unrelated and have fixes merged to the branch: #81544#81391#81848
Required OOB changes look good.
Ready to merge. :shipit:

@carlossanlop
carlossanlop merged commit 0e280c4 into dotnet:release/6.0Feb 9, 2023
@ghostghost locked as resolved and limited conversation to collaborators Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects

None yet

Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync - #80552
Merged
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Feb 9, 2023
Merged
[release/6.0] Enforce HttpClient limits on GetFromJsonAsync#80552
carlossanlop merged 3 commits into
dotnet:release/6.0from 
MihaZupan:backport-json-net6
Conversation
@MihaZupan
@MihaZupanMihaZupan
commented
Jan 12, 2023•

edited by karelz

Loading

Copy link


Copy Markdown

Member
Backport of a minimized change of #79386 to release/6.0
Customer Impact
HttpClient has two properties users can tweak to limit the amount of time and resources spent on a given request (Timeout and MaxResponseContentBufferSize).
GetFromJsonAsync is inconsistent in the enforcement of these limits compared to other helpers (GetStringAsync, GetByteArrayAsync, and DeleteFromJsonAsync).
There are three main ways to get the response content from HttpClient:
  1. Using the one-line helper methods

    awaitclient.GetStringAsync("foo");// Limits enforcedawaitclient.GetFromJsonAsync<MyClass>("foo");// Limits **NOT** enforced
  2. Get the response object and call helpers on its content

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request);awaitresponse.Content.ReadAsStringAsync();// Limits enforcedawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits enforced
  3. Optionally use ResponseHeadersRead, asking the client not to buffer the response content as part of the SendAsync call

    usingHttpResponseMessageresponse=awaitclient.SendAsync(request,HttpCompletionOption.ResponseHeadersRead);awaitresponse.Content.ReadAsStringAsync();// Limits not enforced by designawaitresponse.Content.ReadFromJsonAsync<MyClass>();// Limits not enforced by design
This PR changes the behavior of the client.GetFromJsonAsync helper to match that of GetStringAsync and friends (case 1).

This allows us to present consistent HttpClient behavior across the board.
Testing
I added targeted CI tests that confirm limits are consistently enforced.
Risk
The enforcement of limits means that some requests that would previously succeed may now fail (either time out or exceed the size limit). It is unlikely that anyone is knowingly relying on this behavior given the inconsistencies mentioned above.

The default limits are also very large (100 seconds and 2 GB of content), so for a request to hit them, the user has most likely lowered them manually, indicating the intent that they do want them to be enforced. It also means that if they do run into issues, they can tweak the existing settings directly.
The change can also result in slightly higher memory consumption as we're buffering the whole body before we start the deserialization process. We do not expect this to be meaningfully impactful.
@MihaZupanMihaZupan
added this to the 6.0.x milestone
Jan 12, 2023
@MihaZupan
MihaZupan
requested a review
from a teamJanuary 12, 2023 15:07
@MihaZupanMihaZupan
self-assigned this
Jan 12, 2023
@ghost
ghost
commented
Jan 12, 2023

Copy link


Copy Markdown

Tagging subscribers to this area: @dotnet/ncl

See info in area-owners.md if you want to be subscribed.
Issue Details
Backport of a minimized change of #79386 to release/6.0
Customer Impact
TODO
Testing
TODO
Risk
TODO
Author:MihaZupan
Assignees:MihaZupan
Labels:
area-System.Net.Http
Milestone:6.0.x
@build-analysisbuild-analysisBot
mentioned this pull request
Jan 12, 2023

Closed

@carlossanlop
carlossanlop
commented
Jan 12, 2023

Copy link


Copy Markdown

Contributor
Last day to merge backports for the February Release is tomorrow. Please fill out the template, make sure to mention the customer impact. Add the servicing-release label, and send email to Tactics requesting approval.
@MihaZupanMihaZupan
reopened this
Jan 13, 2023
@carlossanlopcarlossanlop
added
the 
NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
label
Jan 13, 2023
@carlossanlop
carlossanlop
commented
Jan 13, 2023

Copy link


Copy Markdown

Contributor
Talked to @MihaZupan. This will go in next month.
ManickaP
ManickaP
approved these changes
Jan 18, 2023
View reviewed changes
@ManickaPManickaP
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
@MihaZupanMihaZupan
reopened this
Jan 31, 2023
@MihaZupanMihaZupan
mentioned this pull request
Feb 6, 2023

Closed

@MihaZupan
MihaZupan
commented
Feb 6, 2023

Copy link


Copy Markdown

MemberAuthor
Failures are #81391, #81544.
@MihaZupanMihaZupan
added

Servicing-consider
Issue for next servicing release review
and removed

NO-MERGE
The PR is not ready for merge yet (see discussion for detailed reasons)
labels
Feb 8, 2023
@carlossanlop
carlossanlop
commented
Feb 8, 2023

Copy link


Copy Markdown

Contributor
Hey @ViktorHofer what should we do about this failure?:
.dotnet\sdk\6.0.113\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5):
error NETSDK1138: (NETCORE_ENGINEERING_TELEMETRY=Build)
The target framework 'net5.0' is out of support and will not receive security updates in the future.
Please refer to https://aka.ms/dotnet-core-support for more information about the support policy.

@ViktorHofer
ViktorHofer
commented
Feb 8, 2023

Copy link


Copy Markdown

Member
That's happening because of dotnet/sdk@4c7675d. Apparently serviced SDKs now also warn about a TFM that moves out of support, even when that's long after the stable SDK itself shipped originally.
We still produce net5.0 assets in our release/6.0 branch and we don't want to stop doing that. So we need an escape switch =>/p:CheckEolTargetFramework=false.
You can suppress that error by setting <CheckEolTargetFramework>false</CheckEolTargetFramework> here: 
runtime/src/libraries/testPackages/build/Directory.Build.props

Line 5
in
bd54093
 </PropertyGroup> 
@carlossanlopcarlossanlop
mentioned this pull request
Feb 8, 2023
 Merged

@karelz
karelz
commented
Feb 9, 2023

Copy link


Copy Markdown

Member
Approved by Tactics via email by @SteveMCarroll on 2/9.

Adding Servicing-approved label to the PR.
@karelzkarelz
added

Servicing-approved
Approved for servicing release
and removed

Servicing-consider
Issue for next servicing release review
labels
Feb 9, 2023
@carlossanlopcarlossanlop
modified the milestones:
6.0.x, 6.0.15Feb 9, 2023
@carlossanlop
carlossanlop
commented
Feb 9, 2023

Copy link


Copy Markdown

Contributor
Approved by Tactics for 6.0.15.

Signed-off by area owners.

CI failures unrelated and have fixes merged to the branch: #81544#81391#81848

Required OOB changes look good.

Ready to merge. :shipit:
@carlossanlop
carlossanlop
merged commit 0e280c4
into
dotnet:release/6.0Feb 9, 2023
@ghostghost
locked as resolved and limited conversation to collaborators
Mar 11, 2023
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.


Reviewers

@ManickaPManickaPManickaP approved these changes

Assignees

@MihaZupanMihaZupan

Labels

area-System.Net.HttpServicing-approvedApproved for servicing release

Projects


None yet


Milestone

6.0.15

Development

Successfully merging this pull request may close these issues.

5 participants

@MihaZupan@carlossanlop@ViktorHofer@karelz@ManickaP