Description
In .NET7 Linux, we noticed that the Server Side OCSP Stapling is enabled by default, in which the response from OCSP responder is cached to speed up the cert status check
However, if OCSP responder send back a wrong response ...it will be cached as well in TLS server side. Then ,the client of TLS server will always receive the bad extension.status_request , then failed to setup the SSL connection due to parse failure.
From .NET runtime code, I can see the logic to do OCSP downloading , then cache for about 1 day if the response content is not null
https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs#L226C1-L249
Then, inside the download code, I can see the bad http status code is not handled.
https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/System/Net/Http/X509ResourceClient.cs#L182C24-L243
What happens in our TLS server is, it tries to connection a OCSP website, and it should get a 5xx response. The OCSP website configures "an error html page" when it fails, so the response content is not empty, and our TLS server cached the error html page as the OCSP response.
Then, when client side made a https connection to our TLS server, from the ssl debug info, we can see the error html page inside the handshake payload .. It said "Our services aren't available right now
We're working to restore all services as soon as possible. Please check back soon"
08F0: CE 3C 21 44 4F 43 54 59 50 45 20 68 74 6D 6C 20 .<!DOCTYPE html 0900: 50 55 42 4C 49 43 20 27 2D 2F 2F 57 33 43 2F 2F PUBLIC '-//W3C//
0910: 44 54 44 20 58 48 54 4D 4C 20 31 2E 30 20 54 72 DTD XHTML 1.0 Tr
0920: 61 6E 73 69 74 69 6F 6E 61 6C 2F 2F 45 4E 27 20 ansitional//EN' 0930: 27 68 74 74 70 3A 2F 2F 77 77 77 2E 77 33 2E 6F 'http://www.w3.o
0940: 72 67 2F 54 52 2F 78 68 74 6D 6C 31 2F 44 54 44 rg/TR/xhtml1/DTD
0950: 2F 78 68 74 6D 6C 31 2D 74 72 61 6E 73 69 74 69 /xhtml1-transiti
0960: 6F 6E 61 6C 2E 64 74 64 27 3E 3C 68 74 6D 6C 20 onal.dtd'><html 0970: 78 6D 6C 6E 73 3D 27 68 74 74 70 3A 2F 2F 77 77 xmlns='http://ww
0980: 77 2E 77 33 2E 6F 72 67 2F 31 39 39 39 2F 78 68 w.w3.org/1999/xh
0990: 74 6D 6C 27 3E 3C 68 65 61 64 3E 3C 6D 65 74 61 tml'><head><meta
09A0: 20 63 6F 6E 74 65 6E 74 3D 27 74 65 78 74 2F 68 content='text/h
09B0: 74 6D 6C 3B 20 63 68 61 72 73 65 74 3D 75 74 66 tml; charset=utf
09C0: 2D 38 27 20 68 74 74 70 2D 65 71 75 69 76 3D 27 -8' http-equiv='
09D0: 63 6F 6E 74 65 6E 74 2D 74 79 70 65 27 2F 3E 3C content-type'/><
09E0: 73 74 79 6C 65 20 74 79 70 65 3D 27 74 65 78 74 style type='text
09F0: 2F 63 73 73 27 3E 62 6F 64 79 20 7B 66 6F 6E 74 /css'>body .font
0A00: 2D 66 61 6D 69 6C 79 3A 41 72 69 61 6C 3B 20 6D -family:Arial; m
0A10: 61 72 67 69 6E 2D 6C 65 66 74 3A 34 30 70 78 3B argin-left:40px;
0A20: 20 7D 69 6D 67 20 20 7B 20 62 6F 72 64 65 72 3A .img . border:
0A30: 30 20 6E 6F 6E 65 3B 20 7D 23 63 6F 6E 74 65 6E 0 none; .#conten
0A40: 74 20 7B 20 6D 61 72 67 69 6E 2D 6C 65 66 74 3A t . margin-left:
0A50: 20 61 75 74 6F 3B 20 6D 61 72 67 69 6E 2D 72 69 auto; margin-ri
0A60: 67 68 74 3A 20 61 75 74 6F 20 7D 23 6D 65 73 73 ght: auto .#mess
0A70: 61 67 65 20 68 32 20 7B 20 66 6F 6E 74 2D 73 69 age h2 . font-si
0A80: 7A 65 3A 20 32 30 70 78 3B 20 66 6F 6E 74 2D 77 ze: 20px; font-w
0A90: 65 69 67 68 74 3A 20 6E 6F 72 6D 61 6C 3B 20 63 eight: normal; c
0AA0: 6F 6C 6F 72 3A 20 23 30 30 30 30 30 30 3B 20 6D olor: #000000; m
0AB0: 61 72 67 69 6E 3A 20 33 34 70 78 20 30 70 78 20 argin: 34px 0px 0AC0: 30 70 78 20 30 70 78 20 7D 23 6D 65 73 73 61 67 0px 0px .#messag
0AD0: 65 20 70 20 20 7B 20 66 6F 6E 74 2D 73 69 7A 65 e p . font-size
0AE0: 3A 20 31 33 70 78 3B 20 63 6F 6C 6F 72 3A 20 23 : 13px; color: #
0AF0: 30 30 30 30 30 30 3B 20 6D 61 72 67 69 6E 3A 20 000000; margin: 0B00: 37 70 78 20 30 70 78 20 30 70 78 30 70 78 7D 23 7px 0px 0px0px.#
0B10: 65 72 72 6F 72 72 65 66 20 7B 20 66 6F 6E 74 2D errorref . font-
0B20: 73 69 7A 65 3A 20 31 31 70 78 3B 20 63 6F 6C 6F size: 11px; colo
0B30: 72 3A 20 23 37 33 37 33 37 33 3B 20 6D 61 72 67 r: #737373; marg
0B40: 69 6E 2D 74 6F 70 3A 20 34 31 70 78 20 7D 3C 2F in-top: 41px .</
0B50: 73 74 79 6C 65 3E 3C 74 69 74 6C 65 3E 43 45 53 style><title>CES
0B60: 50 4B 49 3C 2F 74 69 74 6C 65 3E 3C 2F 68 65 61 PKI</title></hea
0B70: 64 3E 3C 62 6F 64 79 3E 3C 64 69 76 20 69 64 3D d><body><div id=
0B80: 27 63 6F 6E 74 65 6E 74 27 3E 3C 64 69 76 20 69 'content'><div i
0B90: 64 3D 27 6D 65 73 73 61 67 65 27 3E 3C 68 32 3E d='message'><h2>
0BA0: 4F 75 72 20 73 65 72 76 69 63 65 73 20 61 72 65 Our services are
0BB0: 6E 27 74 20 61 76 61 69 6C 61 62 6C 65 20 72 69 n't available ri
0BC0: 67 68 74 20 6E 6F 77 3C 2F 68 32 3E 3C 70 3E 57 ght now</h2><p>W
0BD0: 65 27 72 65 20 77 6F 72 6B 69 6E 67 20 74 6F 20 e're working to 0BE0: 72 65 73 74 6F 72 65 20 61 6C 6C 20 73 65 72 76 restore all serv
0BF0: 69 63 65 73 20 61 73 20 73 6F 6F 6E 20 61 73 20 ices as soon as 0C00: 70 6F 73 73 69 62 6C 65 2E 20 50 6C 65 61 73 65 possible. Please
0C10: 20 63 68 65 63 6B 20 62 61 63 6B 20 73 6F 6F 6E check back soon
0C20: 2E 3C 2F 70 3E 3C 2F 64 69 76 3E 3C 64 69 76 20 .</p></div><div 0C30: 69 64 3D 27 65 72 72 6F 72 72 65 66 27 3E 3C 73 id='errorref'><s
0C40: 70 61 6E 3E 52 65 66 20 41 3A 20 34 33 30 46 37 pan>Ref A: 430F7
0C50: 35 37 35 34 30 35 44 34 30 36 32 38 32 45 36 44 575405D406282E6D
0C60: 38 41 31 32 38 33 42 43 45 35 44 20 52 65 66 20 8A1283BCE5D Ref 0C70: 42 3A 20 43 48 31 41 41 32 30 34 30 39 30 34 30 B: CH1AA20409040
0C80: 32 35 20 52 65 66 20 43 3A 20 32 30 32 33 2D 30 25 Ref C: 2023-0
0C90: 37 2D 32 35 54 31 38 3A 34 35 3A 32 31 5A 3C 2F 7-25T18:45:21Z</
0CA0: 73 70 61 6E 3E 3C 2F 64 69 76 3E 3C 2F 64 69 76 span></div></div
0CB0: 3E 3C 2F 62 6F 64 79 3E 3C 2F 68 74 6D 6C 3E 00 ></body></html>.
Then with decrypted, client side saw
"extensions": {
"status_request (5)": {
extra data at the end
}
}
and failed with
Caused by: java.io.IOException: extra data at the end
at java.base/sun.security.util.DerValue.<init>(DerValue.java:428)
The issue can be resolved when we restarted our TLS server which will get a refresh ocsp response.
However, I wonder whether it is possible to check the status code of ocsp response, and avoid the caching when the response is not correct?
Reproduction Steps
intermittent issue due to ocsp responder availablity.
maybe setup a bad ocsp responder can reproduce it stably.
Expected behavior
expect the retry on ocsp download or avoid cache the ocsp bad result
Actual behavior
the bad ocsp response is cached.
Regression?
No response
Known Workarounds
restart TLS server to fetch a new OCSP response
Configuration
.NET7 Kershel webserver on Linux
Other information
No response
Drafted a possible fix #89908 for this issue. Please help to review
Description
In .NET7 Linux, we noticed that the Server Side OCSP Stapling is enabled by default, in which the response from OCSP responder is cached to speed up the cert status check
However, if OCSP responder send back a wrong response ...it will be cached as well in TLS server side. Then ,the client of TLS server will always receive the bad extension.status_request , then failed to setup the SSL connection due to parse failure.
From .NET runtime code, I can see the logic to do OCSP downloading , then cache for about 1 day if the response content is not null
https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Security/src/System/Net/Security/SslStreamCertificateContext.Linux.cs#L226C1-L249
Then, inside the download code, I can see the bad http status code is not handled.
https://github.com/dotnet/runtime/blob/main/src/libraries/Common/src/System/Net/Http/X509ResourceClient.cs#L182C24-L243
What happens in our TLS server is, it tries to connection a OCSP website, and it should get a 5xx response. The OCSP website configures "an error html page" when it fails, so the response content is not empty, and our TLS server cached the error html page as the OCSP response.
Then, when client side made a https connection to our TLS server, from the ssl debug info, we can see the error html page inside the handshake payload .. It said "Our services aren't available right now
We're working to restore all services as soon as possible. Please check back soon"
Then with decrypted, client side saw
and failed with
The issue can be resolved when we restarted our TLS server which will get a refresh ocsp response.
However, I wonder whether it is possible to check the status code of ocsp response, and avoid the caching when the response is not correct?
Reproduction Steps
intermittent issue due to ocsp responder availablity.
maybe setup a bad ocsp responder can reproduce it stably.
Expected behavior
expect the retry on ocsp download or avoid cache the ocsp bad result
Actual behavior
the bad ocsp response is cached.
Regression?
No response
Known Workarounds
restart TLS server to fetch a new OCSP response
Configuration
.NET7 Kershel webserver on Linux
Other information
No response
Drafted a possible fix #89908 for this issue. Please help to review