Bad BaseHTTPRequestHandler response when using HTTP/0.9

[zhangyangyu]

BPO	26578
Nosy	@warsaw, @gpshead, @vadmium, @zhangyangyu
Dependencies	bpo-10721: Remove HTTP 0.9 server support
Files	force-0.9.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-03-17.07:42:36.782>
labels = ['type-bug', 'library']
title = 'Bad BaseHTTPRequestHandler response when using HTTP/0.9'
updated_at = <Date 2016-10-29.11:19:59.882>
user = 'https://github.com/zhangyangyu'

bugs.python.org fields:

activity = <Date 2016-10-29.11:19:59.882>
actor = 'xiang.zhang'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2016-03-17.07:42:36.782>
creator = 'xiang.zhang'
dependencies = ['10721']
files = ['45257']
hgrepos = []
issue_num = 26578
keywords = ['patch']
message_count = 6.0
messages = ['261898', '261899', '261949', '261952', '279642', '279671']
nosy_count = 4.0
nosy_names = ['barry', 'gregory.p.smith', 'martin.panter', 'xiang.zhang']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26578'
versions = ['Python 3.6']

Linked PRs

• gh-70765: avoid waiting for HTTP headers when parsing HTTP/0.9 requests #139514
• [3.14] gh-70765: avoid waiting for HTTP headers when parsing HTTP/0.9 requests (GH-139514) #139600
• [3.13] gh-70765: avoid waiting for HTTP headers when parsing HTTP/0.9 requests (GH-139514) #139602
• gh-70765: fix an HTTP/0.9 flaky test post GH-139514 #139610
• gh-70765: Remove unnecessary extra backtick from Changelog entry #140071
• [3.13] gh-70765: Remove unnecessary extra backtick from Changelog entry (GH-140071) #140101
• [3.14] gh-70765: Remove unnecessary extra backtick from Changelog entry (GH-140071) #140102

[zhangyangyu]

BaseHTTPRequestHandler in http.server supports HTTP/0.9. But the response for HTTP/0.9 request is implemented wrong.

Response of HTTP/0.9 request returns message body directly without status line and headers. But if you inherit BaseHTTPRequestHandler and set the default_request_version to "HTTP/1.x", then self.request_version can never be "HTTP/0.9" since in the https://hg.python.org/cpython/file/tip/Lib/http/server.py#l315 branch it does not set version to "HTTP/0.9" and then always sends the status line and headers back.

A trivial patch can fix this problem that set version to "HTTP/0.9" in the branch. But this will cause some failure in tests. The tests in test_httpservers use http.client.HTTPConnection to send and receive HTTP message. But since 3.4, HTTPConnection doesn't support HTTP/0.9-style simple responses. We can use it to test HTTP/0.9 connection if the server is implemented in the right way.

And since http.client.HTTPConnection has dropped the support for HTTP/0.9, is it reasonable to drop the support in http.server too?

[zhangyangyu]

can should be can not.

And not only HTTPConnection, support for HTTP/0.9 seems to have been totally abandoned since Python3.4 in http.client.

[vadmium]

As I understand it, you are saying if you override the undocumented (but publicly-named) default_request_version attribute, HTTP 0.9 requests no longer work.

I suspect it is even broken by default. My understanding is with HTTP 0.9 you should be able to send b"GET <path>\r\n" and get a response, but Python’s server will deadlock waiting for a second blank line or EOF. It looks like this deadlock has been there since ~forever (1995).

See bpo-10721 which already proposes to remove 0.9 server support. I would be weakly in favour of this (or more strongly if someone can prove my theory that the current support is broken).

[zhangyangyu]

I think you are right. Simply run http.server.test() and then telnet to send "GET /", the client hangs. Or add a timeout to BaseHTTPRequestHandler, you can see the timeout error on server output.

[vadmium]

Since my last comment, I have become more confident that Python’s request parsing never supported proper HTTP 0.9. So I would prefer to remove it in the next version of Python, and not bother fixing Xiang’s bug in existing versions (unless someone offers a practical reason to fix it).

For the record, this patch is what a fix might look like.

[zhangyangyu]

So I would prefer to remove it in the next version of Python, and not bother fixing Xiang’s bug in existing versions.

+1. In rfc7230, "The expectation to support HTTP/0.9 requests has been removed".

[picnixz]

There was a simple fix which consisted in not parsing the rest of the request as there shouldn't be headers. But deprecating the support would also be fine (there is a lot of code that is still assuming HTTP/0.9 but it's actually never tested because we never get past parse_request where it hangs because it expects some headers).

Note that HTTPConnection has no support for HTTP/0.9 so a client would need to use raw sockets.