Skip to content

gh-131423: Update OpenSSL data to 3.4.1 on Linux - #131618

Merged
picnixz merged 15 commits into
python:mainfrom
picnixz:ci/update/ssl-versions-131423
Apr 25, 2025
Merged

gh-131423: Update OpenSSL data to 3.4.1 on Linux#131618
picnixz merged 15 commits into
python:mainfrom
picnixz:ci/update/ssl-versions-131423

Conversation

@picnixz

@picnixzpicnixz commented Mar 23, 2025•
edited
Loading

Copy link
Copy Markdown
Member

I've also updated the make_ssl_data.py script that @encukou has recently updated as well. I completed with instructions that I thought usefull for future maintainers.

📚 Documentation preview 📚: https://cpython-previews--131618.org.readthedocs.build/

@picnixz
update OpenSSL CI build configuration
85ec7b2
@picnixz
update OpenSSL mnemonics
40ad7ce
Since mnemonics from 3.4.1 are different (renumbered) from 3.4.0.
To ease future updates, we assume the following:
`_ssl_data_<MAJOR><PATCH>.h` contains the latest OpenSSL data. If the
previous `_ssl_data_<MAJOR><PATCH>.h` file is incompatible with the
newest one (e.g., because some mnemonics were renamed or removed), the
old one is renamed to `_ssl_data_<MAJOR><MINOR><PATCH>.h` where <PATCH>
is the patch number it was based upon.
In this commit, OpenSSL 3.4.1 mnemonics are not compatible with OpenSSL
3.4.0 mnemonics as they were renumbered. Therefore, `_ssl_data_34.h` is
renamed to `_ssl_data_340.h` and `_ssl_data_34x.h` now contains OpenSSL
3.4.1 mnemonics.
We also refined the mnemonics that are selected, discarding those that
are mnemonics-like but should not be used as such (e.g., ERR_LIB_MASK
and ERR_LIB_OFFSET for OpenSSL 1.1.1).
@picnixz
update macOS and Windows OpenSSL configurations
b35b7fa
@picnixz
Merge remote-tracking branch 'upstream/main' into ci/update/ssl-versi…
578bb0e
…ons-131423
@picnixz
update SBOM
296a98c
@picnixz
update SBOM
326d947
@picnixz
update configure requirements
cf7a65d
@bedevere-appbedevere-appBot mentioned this pull request Mar 23, 2025
Closed
4 tasks
@picnixz
picnixzforce-pushed the ci/update/ssl-versions-131423 branch from 05ee142 to 5bbc702CompareMarch 23, 2025 10:53
@picnixz
picnixz marked this pull request as ready for review March 24, 2025 14:56
encukou
encukou reviewed Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py
Comment on lines +146 to +147
# FEAT(picnixz): in the future, we may want to also check
# the consistency of the OpenSSL files with an external tool.

@encukouencukouMar 24, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you track that in an issue instead?

@picnixzpicnixzApr 20, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in #132745.

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py Outdated
Comment threadTools/ssl/make_ssl_data.py
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py Outdated
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py Outdated
@picnixz
picnixz marked this pull request as draft March 24, 2025 17:49
@picnixz

picnixz commented Mar 24, 2025

Copy link
Copy Markdown
MemberAuthor

arf, I'm not on my Linux so I can't regen :< I'm leaving tomorrow morning so I'm not really sure I'll be able to commit before leaving, but otherwise, just take over the PR and regen the data!

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
Comment threadTools/ssl/make_ssl_data.py Outdated
ned-deily
ned-deily reviewed Mar 26, 2025
View reviewed changes

@ned-deilyned-deily left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macOS build-installer.py changes LGTM, thanks!

@picnixz
picnixz requested a review from encukouMarch 28, 2025 22:50
@picnixz
picnixz requested a review from gpshead as a code ownerApril 5, 2025 10:52
@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS, Windows) and 3.4.1 (Linux)gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux)Apr 5, 2025
@picnixzpicnixz added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@bedevere-bot

bedevere-bot commented Apr 5, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 905f1a5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131618%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@picnixz

picnixz commented Apr 5, 2025

Copy link
Copy Markdown
MemberAuthor

To avoid surprises, I'm running the build bots. If they pass, I'll merge this one so that we can close the other issue. I think the Windows-related failures were recently solved as well

@picnixz

picnixz commented Apr 5, 2025•
edited by hugovk
Loading

Copy link
Copy Markdown
MemberAuthor

The iOS failure is known (PR #132050)

@picnixz

picnixz commented Apr 6, 2025

Copy link
Copy Markdown
MemberAuthor

I want to think about something. Mnenmonics were updated in 3.4.1 compared to 3.4.0, but that's only because I knew that they were changed. However, we're actually having a

#if (OPENSSL_VERSION_NUMBER >= 0x30100000L)
#include"_ssl_data_34.h"

So I think I'll need a way to check first that when OpenSSL mnemonics changed so that we regenerate the correct files per version.

@ned-deily

ned-deily commented Apr 6, 2025

Copy link
Copy Markdown
Member

@picnixz, with the 3.14.a7, 3.13.3, and 3.12.10 releases approaching in two days, I plan to update the macOS installers for those releases to use 3.0.16. If you don't expect to be able to merge this PR before then, I can pull out the build-installer.py change into a separate PR since it has no relation to any of the other changes in this PR. (And that's why I prefer to keep changes like this separate.)

@picnixz

picnixz commented Apr 6, 2025

Copy link
Copy Markdown
MemberAuthor

If you don't expect to be able to merge this PR before then

To be on the safe side, please do so. I don't want to block the macOS side with my interrogations. Hopefully I'll be able to merge this before the release.

ned-deily
ned-deily reviewed Apr 7, 2025
View reviewed changes
Comment threadMac/BuildScript/build-installer.py
ned-deily
ned-deily reviewed Apr 7, 2025
View reviewed changes
Comment threadMisc/NEWS.d/next/macOS/2025-03-23-11-32-09.gh-issue-131423.s1Lvli.rst Outdated
@picnixz

picnixz commented Apr 7, 2025•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Note: 3.4.1 includes CVE patches but since we're still in alpha, we can say that those security patches are not really necessary now (in addition, they affect components that are not directly exposed by Python IIRC). So, I'll postpone this until after the release (I don't want to have surprises where a mnemonic change would be actually annoying for a user)

@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux)gh-131423: Update OpenSSL to 3.4.1 on LinuxApr 7, 2025
@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.4.1 on Linuxgh-131423: Update OpenSSL data to 3.4.1 on LinuxApr 8, 2025
@picnixz

picnixz commented Apr 25, 2025

Copy link
Copy Markdown
MemberAuthor

I'll merge this one and work on #132745.

@picnixz
picnixz merged commit 6a9bfee into python:mainApr 25, 2025
@picnixz
picnixz deleted the ci/update/ssl-versions-131423 branch April 25, 2025 08:27
@picnixz

picnixz commented Apr 25, 2025

Copy link
Copy Markdown
MemberAuthor

Rationale for not backporting: #131423 (comment).

@zwarezware mentioned this pull request Aug 14, 2025
Merged
@encukouencukou mentioned this pull request Aug 15, 2025
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zoobazoobazooba left review comments
@ned-deilyned-deilyned-deily left review comments
@gpsheadgpsheadgpshead approved these changes
@erlend-aaslanderlend-aaslandAwaiting requested review from erlend-aaslanderlend-aasland is a code owner
@corona10corona10Awaiting requested review from corona10corona10 is a code owner
@ezio-melottiezio-melottiAwaiting requested review from ezio-melottiezio-melotti is a code owner
@hugovkhugovkAwaiting requested review from hugovkhugovk is a code owner
@AA-TurnerAA-TurnerAwaiting requested review from AA-TurnerAA-Turner is a code owner
@encukouencukouAwaiting requested review from encukou

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@picnixz@bedevere-bot@ned-deily@gpshead@encukou@zooba