[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

[release/7.0] Fix server-side OCSP stapling on Linux - #96808

Merged
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0
Jan 16, 2024
Merged

[release/7.0] Fix server-side OCSP stapling on Linux#96808
rzikm merged 7 commits into
dotnet:release/7.0-stagingfrom
rzikm:ocsp-verify-fix-7.0

Conversation

@rzikm

@rzikmrzikm commented Jan 10, 2024

Copy link
Copy Markdown
Member

Backport of PR #96792, PR #96448, PR #90200 and PR #96972.

Fixes#96770, #96659 and #89907

Description

Regression: No, .NET 6 didn't have OCSP staple feature.
Customer: Internal partner team - blocking migration from Windows to Linux.

OCSP (Online Certificate Status Protocol) stapling is an optimization where instead of clients individually retrieving revocation status of the server certificate, server will fetch the OCSP response itself and send it to clients during connection handshake. The authenticity of the response is assured by a digital signature.

First bug in .NET 7.0+ ... An invalid response can get cached and the server would fail to refresh it. The server would then keep sending the old, cached and potentially malformed OCSP response. This bug is triggered when either:

Second bug in .NET 7.0+ ... Validation of OCSP response always fails when the method of "delegated signing" is used (delegated signing means that the OCSP response is signed by a special certificate delegated by the server certificate issuer).

  • Note that validation when receiving OCSP response on client-side is not affected by this bug and is extensively tested, only the server part is affected by the bug.
  • Server validating the OCSP response before sending it out is only to play nice and not send invalid OCSP responses out. This bug DOES NOT create security vulnerability because clients are expected to independently validate the OCSP response themselves.
    Fixed in main by PR Add entire issuer chain to trusted X509_STORE when stapling OCSP_Response #96792

The two issues mentioned above may lead to following undesired behaviors:

  • Server caches an invalid OCSP staple (e.g. error page due to OCSP server outage), and keeps sending it out.
  • Server stops refreshing the OCSP staple and keeps sending the old one even after its validity expired.
  • Server sends out OCSP staple which it does not consider valid

Customer Impact

Android clients cannot connect to .NET 7+ servers affected by this bug because the OCSP information may get outdated (and not refreshed) and Android 9+'s application default security restrictions don't allow HTTP connections by default.

Regression

No, sending OCSP staples from .NET servers is a new feature in .NET 7.

Testing

Locally reproduced affected scenario and extensively tested manually.

Customer validated private 7.0 bits.

There is extensive existing test coverage on client-side OCSP usage/validation. Missing E2E server-side automated test coverage is planned for upcoming weeks.

Risk

Small to medium. Code touched by this PR is not used in other code paths than OCSP, so only "Sending OCSP staples from a server" scenario is affected.

@ghostghost assigned rzikmJan 10, 2024
@rzikm
rzikm changed the base branch from main to release/7.0-stagingJanuary 10, 2024 19:59
@ghost

Copy link
Copy Markdown

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Reflection.Metadata, new-api-needs-documentation

Milestone:-

@rzikmrzikm changed the title Ocsp-verify-fix-7.0[release/7.0] Fix server-side OCSP stapling on LinuxJan 10, 2024
@ghost

Copy link
Copy Markdown

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes Issue

main PR

Description

Customer Impact

Regression

Testing

Risk

Package authoring signed off?

IMPORTANT: If this change touches code that ships in a NuGet package, please make certain that you have added any necessary package authoring and gotten it explicitly reviewed.

Author:rzikm
Assignees:rzikm
Labels:

area-System.Net.Security, new-api-needs-documentation

Milestone:-

@rzikm

Copy link
Copy Markdown
MemberAuthor

No *.ref assemblies were touched by this PR

rzikmand others added 4 commits January 11, 2024 23:02
* Recover from failed OCSP check.
* Add 5s back-off after failed OCSP querry
…sponse
Code review feedback
More code review feedback
Update src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Fix compilation
Always include root certificate
@rzikm
rzikmforce-pushed the ocsp-verify-fix-7.0 branch from bc58611 to 1efdf2bCompareJanuary 11, 2024 22:02
@carlossanlop

Copy link
Copy Markdown
Contributor

There was a generalized failure in the 7.0 branch which impacted this PR. I'm updating the branch so that your PR gets rebased to the latest bits.

@rzikm
rzikm marked this pull request as ready for review January 15, 2024 08:26
@rzikm
rzikm requested a review from bartonjsJanuary 15, 2024 08:26
@karelzkarelz added the Servicing-consider Issue for next servicing release review label Jan 15, 2024
@rzikm

Copy link
Copy Markdown
MemberAuthor

Build analysis shows that CI failures are either known or unrelated.

@karelzkarelz added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Jan 16, 2024
@karelz

Copy link
Copy Markdown
Member

Approved by Tactics (@SteveMCarroll) on 1/15 via email - label updated to Servicing-approved.

@carlossanlop

Copy link
Copy Markdown
Contributor

@karelz@rzikm who can provide a sign-off? Are the CI failures unrelated?

@rzikm

Copy link
Copy Markdown
MemberAuthor

CI failures are not related to the changes.

For code review I would like to wait for @bartonjs (together with the 8.0 PR).

We are also considering adding #96972 to this, it's a oneliner which fixes a very unlikely scenario. I know this is last-minute, but this change would probably not pass servicing on its own.

@rzikm
rzikm requested a review from wfurtJanuary 16, 2024 19:25

@wfurtwfurt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

* Don't shorten OCSP expriation on failed server OCSP fetch
* Code review feedback
@rzikm
rzikm merged commit 7a97ad4 into dotnet:release/7.0-stagingJan 16, 2024
@github-actionsgithub-actionsBot locked and limited conversation to collaborators Feb 16, 2024
@karelzkarelz modified the milestones: 7.0.x, 7.0.16Jun 25, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.Net.Securityos-linuxLinux OS (any supported distro)Servicing-approvedApproved for servicing release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@rzikm@carlossanlop@karelz@bartonjs@wfurt@vcsjones