gh-132983: Don't allow trailer data in ZstdFile

[Rogdham]

We previously made sure that an exception is raised when decompressing trailer data with decompress:

>>>fromcompression.zstdimportcompress, decompress>>>invalid=compress(b'xxx') +b'yyy'>>>decompress(invalid)
Traceback (mostrecentcalllast):
File"<python-input-2>", line1, in<module>decompress(invalid)
~~~~~~~~~~^^^^^^^^^File"/redacted/Lib/compression/zstd/__init__.py", line157, indecompressresults.append(decomp.decompress(data))
~~~~~~~~~~~~~~~~~^^^^^^_zstd.ZstdError: Unabletodecompresszstddata: Unknownframedescriptor

Indeed, the Zstandard specification says “Zstandard compressed data is made of one or more frames”, and it does not say that random data can be added at the end.

However, this is not the case in ZstdFile / zstd.open:

>>>fromcompression.zstdimportZstdFile>>>fromioimportBytesIO>>>ZstdFile(BytesIO(invalid)).read()
b'xxx'

After this PR, the last call becomes:

>>>ZstdFile(BytesIO(invalid)).read()
Traceback (mostrecentcalllast):
File"<python-input-5>", line1, in<module>ZstdFile(BytesIO(invalid)).read()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^File"/redacted/Lib/compression/zstd/_zstdfile.py", line176, inreadreturnself._buffer.read(size)
~~~~~~~~~~~~~~~~~^^^^^^File"/redacted/Lib/compression/_common/_streams.py", line118, inreadallwhiledata:=self.read(sys.maxsize):
~~~~~~~~~^^^^^^^^^^^^^File"/redacted/Lib/compression/_common/_streams.py", line91, inreaddata=self._decompressor.decompress(rawblock, size)
_zstd.ZstdError: Unabletodecompresszstddata: Unknownframedescriptor

---

• Issue: Implement PEP 784 - Adding Zstandard to the Python standard library #132983

[emmatyping]

The current behavior matches LZMA. I think unlike decompress which is handed what is necessarily a zstd stream of one or more frames, with ZstdFile, a user may be parsing a format which has additional information after a zstd stream.

>>> from lzma import LZMAFile, compress
>>> from io import BytesIO
>>> invalid = compress(b'foo') +b'bar'
>>> LZMAFile(BytesIO(invalid)).read()
b'foo'
>>>

[Rogdham]

You are right this is the case for LZMAFile with format FORMAT_AUTO (which is the default) and also for BZ2File.

However, LZMAFile with format FORMAT_XZ as well as GzipFile raise an exception in that case.

>>>fromlzmaimportLZMAFile, compress, FORMAT_XZ>>>fromioimportBytesIO>>>invalid=compress(b'foo') +b'bar'>>>LZMAFile(BytesIO(invalid), format=FORMAT_XZ).read()
Traceback (mostrecentcalllast):
File"<python-input-3>", line1, in<module>LZMAFile(BytesIO(invalid), format=FORMAT_XZ).read()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^File"/redacted/lzma.py", line208, inreadreturnself._buffer.read(size)
~~~~~~~~~~~~~~~~~^^^^^^File"/redacted/_compression.py", line118, inreadallwhiledata:=self.read(sys.maxsize):
~~~~~~~~~^^^^^^^^^^^^^File"/redacted/_compression.py", line99, inreadraiseEOFError("Compressed file ended before the ""end-of-stream marker was reached")
EOFError: Compressedfileendedbeforetheend-of-streammarkerwasreached

[emmatyping]

Okay this looks good then!

[Rogdham]

In addition, consider decompress(compress(b"xxx") + b"yyy"):

• returns b"xxx" on: lzma (format FORMAT_AUTO), bz2
• raises an exception on: lzma (format FORMAT_XZ), gzip

Since for zstd we raise an exception on that, I would say to do the same for ZstdFile to be consistent.

[miss-islington-app]

Thanks @Rogdham for the PR, and @AA-Turner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-133799 is a backport of this pull request to the 3.14 branch.