Skip to content

gh-91349: Adjust default compression level to 6 (down from 9) in gzip and tarfile - #131470

Merged
gpshead merged 8 commits into
python:mainfrom
man-group:rmorotti-compresion-level
Jul 20, 2025
Merged

gh-91349: Adjust default compression level to 6 (down from 9) in gzip and tarfile#131470
gpshead merged 8 commits into
python:mainfrom
man-group:rmorotti-compresion-level

Conversation

@morotti

@morottimorotti commented Mar 19, 2025

Copy link
Copy Markdown
Contributor

It is the default level used by most compression tools and a better tradeoff between speed and performance.

see discussion in #91349 (comment)


📚 Documentation preview 📚: https://cpython-previews--131470.org.readthedocs.build/

@merwokmerwok added awaiting core review stdlib Standard Library Python modules in the Lib/ directory type-feature A feature request or enhancement and removed awaiting core review labels May 19, 2025
@morotti
morottiforce-pushed the rmorotti-compresion-level branch from e2d0562 to a2d0966CompareJune 25, 2025 13:22
…n gzip and tarfile
It is the default level used by most compression tools and a
better tradeoff between speed and performance.
@morotti
morottiforce-pushed the rmorotti-compresion-level branch from a2d0966 to b17e21dCompareJune 25, 2025 13:31
Comment threadDoc/library/gzip.rst
@morotti

Copy link
Copy Markdown
ContributorAuthor

@merwok I added the versionchanged notes. and found another comment in tarfile.rst to adjust.
it's a bit repetitive to add on each signature I suppose but I suppose it's fine.

You can review again.

@emmatypingemmatyping removed type-feature A feature request or enhancement stdlib Standard Library Python modules in the Lib/ directory labels Jun 26, 2025
@morotti

Copy link
Copy Markdown
ContributorAuthor

@gpshead you've replied multiple times to my questions on improving compression code with "PR welcome". maybe you would be interested in reviewing this PR?

@morotti

Copy link
Copy Markdown
ContributorAuthor

can anybody review the PR please?

Comment threadDoc/library/gzip.rst Outdated
Comment threadDoc/library/gzip.rst Outdated
Comment threadDoc/library/gzip.rst Outdated
@morotti

Copy link
Copy Markdown
ContributorAuthor

are you two saying that this fix will not be able to ship with python 3.14?

set versionchanged to next
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Comment threadDoc/library/gzip.rst Outdated
set versionchanged to next
Comment threadDoc/library/gzip.rst Outdated
Comment threadDoc/library/tarfile.rst Outdated
set versionchanged to next
@morotti

Copy link
Copy Markdown
ContributorAuthor

done, adjusted all the versionchanged to next

@merwokmerwok added the 3.15 pre-release feature fixes, bugs and security fixes label Jul 18, 2025
@gpshead

Copy link
Copy Markdown
Member

wow, checking the source history it looks like "9" was used when the gzip module was added circa 1997. ironic, i'd have never expected that as a default. "6" being the bare "gzip" default value is just a well known thing.

Comment threadMisc/NEWS.d/next/Library/2025-03-19-12-41-42.gh-issue-91349.8eTOCP.rst Outdated
Comment threadMisc/NEWS.d/next/Library/2025-03-19-12-41-42.gh-issue-91349.8eTOCP.rst Outdated
@morotti

Copy link
Copy Markdown
ContributorAuthor

@ethanfurman the PR says it's "pending review" from you, can you approve?

@gpshead I adjusted the rst to highlight the function, as per the code suggestion.

Indeed, the very first commit from April 1997 set the compression level to 9. It's been wrong for 28 years. :D

I first came across this bug about 9 years ago, when using tornado (web framework). It took whole seconds to compress HTTP responses with the gzip=9 default and slow down every request (whole seconds of delay to compress a couple MB). Tornado was patched to set compresslevel=6 but I never came around to fix the interpreter until now. This PR is the fix of a lifetime :D

@gpshead
gpshead merged commit e24c66d into python:mainJul 20, 2025
41 checks passed
@merwok

Copy link
Copy Markdown
Member

It seems that I misinterpreted the status of 3.14: rc phase started today, we were in beta before. Would this change (accepted 2 days ago) have been ok for 3.14 @hugovk ?

@gpshead

Copy link
Copy Markdown
Member

Entirely up to Hugo. I personally wouldn't bother trying to put this in 3.14 in the rc phase. The perhaps unusual default has been that way for decades now so there is no urgency in changing it. People who want speed are already passing their own compresslevel, or when possible, using a far more modern algorithm than venerable zlib based gzip.

taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
…n gzip and tarfile (pythonGH-131470)
pythongh-91349: Adjust default compression level to 6 (down from 9) in gzip and tarfile
It is the default level used by most compression tools and a
better tradeoff between speed and performance.
Co-authored-by: rmorotti <romain.morotti@man.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
@hugovk

Copy link
Copy Markdown
Member

Yeah, let's keep it for 3.15.

Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
…n gzip and tarfile (pythonGH-131470)
pythongh-91349: Adjust default compression level to 6 (down from 9) in gzip and tarfile
It is the default level used by most compression tools and a
better tradeoff between speed and performance.
Co-authored-by: rmorotti <romain.morotti@man.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.15pre-release feature fixes, bugs and security fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@morotti@gpshead@merwok@hugovk@eendebakpt@emmatyping@rmmancom