Incompatibility with `urllib3>=2.6.0`

[mfriedy]

The latest https://github.com/urllib3/urllib3/releases/tag/2.6.0 has changed the ContentDecoder class, such that the existing Gzip/Brotli decoders are incompatible and throw an exception. From the release notes: If you use custom decompressors, please make sure to update them to respect the changed API of urllib3.response.ContentDecoder.

Problematic code:

python-storage/google/cloud/storage/_media/requests/download.py

Lines 699 to 766 in 6ed12b9

	class _GzipDecoder(urllib3.response.GzipDecoder):
	"""Custom subclass of ``urllib3`` decoder for ``gzip``-ed bytes.
	Allows a checksum function to see the compressed bytes before they are
	decoded. This way the checksum of the compressed value can be computed.
	Args:
	checksum (object):
	A checksum which will be updated with compressed bytes.
	"""
	def __init__(self, checksum):
	super().__init__()
	self._checksum = checksum
	def decompress(self, data):
	"""Decompress the bytes.
	Args:
	data (bytes): The compressed bytes to be decompressed.
	Returns:
	bytes: The decompressed bytes from ``data``.
	"""
	self._checksum.update(data)
	return super().decompress(data)
	# urllib3.response.BrotliDecoder might not exist depending on whether brotli is
	# installed.
	if hasattr(urllib3.response, "BrotliDecoder"):
	class _BrotliDecoder:
	"""Handler for ``brotli`` encoded bytes.
	Allows a checksum function to see the compressed bytes before they are
	decoded. This way the checksum of the compressed value can be computed.
	Because BrotliDecoder's decompress method is dynamically created in
	urllib3, a subclass is not practical. Instead, this class creates a
	captive urllib3.requests.BrotliDecoder instance and acts as a proxy.
	Args:
	checksum (object):
	A checksum which will be updated with compressed bytes.
	"""
	def __init__(self, checksum):
	self._decoder = urllib3.response.BrotliDecoder()
	self._checksum = checksum
	def decompress(self, data):
	"""Decompress the bytes.
	Args:
	data (bytes): The compressed bytes to be decompressed.
	Returns:
	bytes: The decompressed bytes from ``data``.
	"""
	self._checksum.update(data)
	return self._decoder.decompress(data)
	def flush(self):
	return self._decoder.flush()
	else: # pragma: NO COVER
	_BrotliDecoder = None # type: ignore # pragma: NO COVER

[chandra-siri]

Hi @mfriedy ,

Thanks for reporting it! We're working on releasing the fix for the same.

if you're affected by it, please consider using urllib3<=2.5.0 for mitigations

[mfriedy]

Great, yep that was my solution.

I think its just adding the max_length parameters to those two decompress() function signatures, right?

[ErwinJapie]

Same issue here, downgrading urllib3 bypassed the issue for me for now

[chandra-siri]

this PR #1658 should fix the problem.

[chandra-siri]

Hi all,

The issue is fixed now. Please download the latest version of SDK v3.7.0 pip install google-cloud-storage or pip install google-cloud-storage==3.7.0 to get the fix!

Thank you all for your patience!!