Security hole in wsgiref.headers.Headers

[FelixGrbert]

BPO	11671
Nosy	@pjeby, @vstinner, @tiran, @vadmium, @epicfaace
PRs	bpo-11671: add header validation from http.client to wsgiref.headers.Headers #15299
Files	header_newlines_tip.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-03-25.12:14:57.540>
labels = ['type-security', '3.8', '3.7', 'library', '3.9']
title = 'Security hole in wsgiref.headers.Headers'
updated_at = <Date 2019-08-15.04:22:13.409>
user = 'https://bugs.python.org/FelixGrbert'

bugs.python.org fields:

activity = <Date 2019-08-15.04:22:13.409>
actor = 'epicfaace'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2011-03-25.12:14:57.540>
creator = 'Felix.Gr\xc3\xb6bert'
dependencies = []
files = ['29238']
hgrepos = []
issue_num = 11671
keywords = ['patch']
message_count = 10.0
messages = ['132080', '132132', '132393', '182766', '182781', '182782', '182973', '182976', '195487', '195491']
nosy_count = 8.0
nosy_names = ['pje', 'vstinner', 'christian.heimes', 'Arfrever', 'devin', 'Felix.Gr\xc3\xb6bert', 'martin.panter', 'epicfaace']
pr_nums = ['15299']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue11671'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9']

[FelixGrbert]

As noted by security@python.org's response I'm filing this bug here.

In wsgiref.headers.Headers it is possible to include headers which
contain a newline (i.e. \n or \r) either through add_header or
__init__. It is not uncommon that developers provide web applications
to the public in which the HTTP response headers are not filtered for
newlines but are controlled by the user. In such scenarios a malicious
user can use a newline to inject another header or even initiate a
HTTP response body. The impact would be at least equivalent to XSS.
Therefore, I suggest to filter/warn/except header tuples which contain
the above characters upon assignment in wsgiref.headers.

[pjeby]

It is not uncommon that developers provide web applications
to the public in which the HTTP response headers are not filtered for
newlines but are controlled by the user.

Really? Which applications, and which response headers?

Therefore, I suggest to filter/warn/except header tuples which contain
the above characters upon assignment in wsgiref.headers.

Applications that send them are not WSGI compliant anyway, since the spec forbids control characters in header strings -- and wsgiref.validate already validates this.

Still, I'm not aware of any legitimate use case for apps sending user input as an HTTP header where the data wouldn't already be escaped in some fashion -- cookies, URLs, ...?

[FelixGrbert]

If the spec forbids control characters in headers, the module should
enforce that.

The most frequent example of header injection is the redirect-case: an
application is forwarding using the Location header to a user-supplied
URL.
http://google.com/codesearch?as_q=self.redirect%5C%28self.request.get
Other examples are proxies, setting user-agent, or, as you mention,
custom set-cookies headers.

[devin]

Should now be compliant with this part of the spec:

"Each header_value must not include any control characters, including carriage returns or linefeeds, either embedded or at the end. (These requirements are to minimize the complexity of any parsing that must be performed by servers, gateways, and intermediate response processors that need to inspect or modify response headers.)"

[devin]

backported patch to 2.7

[devin]

backported patch to 2.6

[vstinner]

+ if bad_header_value_re.search(_value):
+ error_str = "Bad header value: {0!r} (bad char: {1!r})"
+ raise AssertionError(error_str.format(
+ _value, bad_header_value_re.search(_value).group(0)))

Why do you search the character twice? You can do something like:

match = bad_header_value_re.search(_value)
if match is not None:
 ... match..group(0) ...

Why do you only check value? You should also check _params:

parts = "; ".join(parts)
match = bad_header_value_re.search(parts)
...

And you should also check the name.

Should we do the same checks in httplib?

[devin]

The spec doesn't say anything about the header name. It probably should though, as the same issue exists there.

I used two searches because that's how it's done in wsgiref.validate, and it's not a huge deal to do that because the second one will only execute when there's an error. That said, I changed it to how you proposed.

Here's another stab at that patch.

[tiran]

What do the RFCs for RFC-822 and HTTP 1.1 say about \r and \n in header names?

[devin]

It looks like it's allowed for header line continuation.

http://www.ietf.org/rfc/rfc2616.txt

HTTP/1.1 header field values can be folded onto multiple lines if the
continuation line begins with a space or horizontal tab. All linear
white space, including folding, has the same semantics as SP. A
recipient MAY replace any linear white space with a single SP before
interpreting the field value or forwarding the message downstream.

...

A CRLF is allowed in the definition of TEXT only as part of a header
field continuation. It is expected that the folding LWS will be
replaced with a single SP before interpretation of the TEXT value.