`gzip._GzipReader.read()` with sometimes uninitialized variable

[maurycy]

Bug report

Bug description:

This one is easy to spot but hard to reproduce:

cpython/Lib/gzip.py

Line 589 in d7dbde8

if buf == b"":

buf is initialized only when self._decompressor.needs_input:

cpython/Lib/gzip.py

Line 577 in d7dbde8

buf = self._fp.read(READ_BUFFER_SIZE)

It exists since gh-95534 (#97664).

I spent a lot of time trying to reproduce this bug using the standard zlib decompressor without any success:

import gzip
import io

class EvilDecomp:
 def __init__(self, **kwargs):
 self.needs_input = False
 self.unused_data = b""
 self.eof = False
 def decompress(self, data, max_length):
 self.needs_input = True
 return b""

gzip.zlib._ZlibDecompressor = EvilDecomp

with gzip.GzipFile(
 fileobj=io.BytesIO(gzip.compress(b"hello world")), mode="rb"
) as f:
 f.read(1)

results in:

22:16:40.678647000PM CEST maurycy@gimel /Users/maurycy/src/cpython % ./python.exe meow.py
Traceback (most recent call last):
 File "/Users/maurycy/src/cpython/meow.py", line 18, in <module>
 f.read(1)
 ~~~~~~^^^
 File "/Users/maurycy/src/cpython/Lib/gzip.py", line 349, in read
 return self._buffer.read(size)
 ~~~~~~~~~~~~~~~~~^^^^^^
 File "/Users/maurycy/src/cpython/Lib/compression/_common/_streams.py", line 68, in readinto
 data = self.read(len(byte_view))
 File "/Users/maurycy/src/cpython/Lib/gzip.py", line 589, in read
 if buf == b"":
 ^^^
UnboundLocalError: cannot access local variable 'buf' where it is not associated with a value

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-137571: Fix UnboundLocalError in gzip._GzipReader.read() #137572
• gh-137571: Protect against possible UnboundLocalError in gzip._GzipReader.read() #150222
• [3.15] gh-137571: Protect against possible UnboundLocalError in gzip._GzipReader.read() (GH-150222) #150229
• [3.14] gh-137571: Protect against possible UnboundLocalError in gzip._GzipReader.read() (GH-150222) #150230
• [3.13] gh-137571: Protect against possible UnboundLocalError in gzip._GzipReader.read() (GH-150222) #150231

[ZeroIntensity]

Just for clarification, does this cause an actual problem in practice? The fix seems simple enough that it's probably worth doing, but I want to make sure this isn't just from fuzzing the private API.

[maurycy]

@ZeroIntensity Nope! Neither. I was reviewing various modules to see how similar are different stream implementations across the stdlib. Many classes provide streaming. They employ different strategies (some copy, some use memoryview etc.) but overlap, and I was curious if there's any opportunity to contribute. That's how I found gh-137524 also.

[serhiy-storchaka]

Thank you for your report @maurycy.