Recover from failed OCSP download.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.

[rzikm]

Fixes#96770

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task. As a consequence, the server will stop sending OCSP staples after the failure.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When performing OCSP stapling by server and we get invalid response from the OCSP server (either because we don't get any data back or parsing the OCSP response fails), the server will never try to fetch OCSP again because the SslStreamCertificateContext._pendingDownload field still contains the previous task.

This PR moves the _pendingDownload = null assignment to a place where it is guaranteed to execute.

This has a potential pitfall when the given OCSP server is unhealthy (and consistently returning bad response), then we would potentially issue many requests to it. But I understand that if server does not send OCSP staple then clients may fetch the staple themselves, so I am not 100% sure if mitigation from our side changes much. @bartonjs, @vcsjones, what do you think? If we should mitigate it, what mechanism do you suggest? exponential back-off, or maybe even a simple rate-limiting of max 1 request per x minutes?

Author:	rzikm
Assignees:	-
Labels:	area-System.Net.Security
Milestone:	-

[danmoseley]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

[rzikm]

Very naive question - do servers ever request these from other servers, and if so, so they use this code? Just curious whether this is a case where Kestrel may have its own policy.

if you mean if this is used in "server-to-server" scenarios, then yes. Technically, the initiator of the connection still acts the role of a client. and the receiver of the connection (the machine acting the server role) will run this code and send an OCSP staple if OCSP stapling was enabled (by SslStreamCertificateContext.Create with right parameters).

Clients are requesting OCSP staple from the server by adding a specific extension to the ClientHello when initiating the connection. Right now, there is no API to control this extension from .NET (not even sure if it is present on all underlying platforms).

[wfurt]

LGTM

[wfurt]

BTW do we have test gap? With 5s delay we should be able to craft a test, right?

[rzikm]

The testing is not going to be straightforward because most of the code is private. I filed #96791 to track test coverage and will look into it some more once we fix the more urgent issues.