gh-127330: Update for OpenSSL 3.4 by encukou · Pull Request #127331 · python/cpython

encukou · 2024-11-27T13:20:00Z

Add `git describe` output to headers generated by `make_ssl_data.py`

IMO, this info is more important than the date when the file was generated.
It does mean that the tool now requires a Git checkout of OpenSSL, not for example a release tarball.

I've regenerated the older file to add the info. To the other older file I added a note about manual edits.

Add notes on how to add a new OpenSSL version

I believe that what I wrote is the correct process. ssl experts, do you want to you verify? @jackjansen, @tiran, @dstufft, @alex

Add 3.4 error messages and multissl tests

Hopefully we'll start getting fewer [SYS] unknown error in tests! (See #127257)

Issue: Add support for OpenSSL 3.4 #127330

Also: - Avoid the deprecated `datetime.datetime.utcnow()` - Fix newline This requires that the OpenSSL source tree is a Git checkout, and that you have Git installed. Both should be fine for a tool run manually by people who can change the tool.

encukou · 2024-11-27T13:46:23Z

!buildbot Arch

bedevere-bot · 2024-11-27T13:46:27Z

🤖 New build scheduled with the buildbot fleet by @encukou for commit ea72022 🤖

The command will test the builders whose names match following regular expression: Arch

The builders matched are:

AMD64 Arch Linux Perf PR
AMD64 Arch Linux Asan PR
AMD64 Arch Linux Valgrind PR
aarch64 Fedora Rawhide LTO PR
AMD64 Arch Linux Usan PR
aarch64 RHEL8 LTO PR
AMD64 Arch Linux TraceRefs PR
aarch64 Fedora Stable Clang PR
aarch64 Ubuntu 22.04 BigMem PR
aarch64 CentOS9 LTO + PGO PR
aarch64 Fedora Stable LTO + PGO PR
aarch64 Fedora Stable PR
aarch64 Fedora Rawhide Clang Installed PR
aarch64 CentOS9 LTO PR
aarch64 Fedora Rawhide LTO + PGO PR
aarch64 Fedora Stable Refleaks PR
aarch64 RHEL8 Refleaks PR
aarch64 Fedora Rawhide NoGIL refleaks PR
AMD64 Arch Linux Usan Function PR
aarch64 Fedora Rawhide Clang PR
aarch64 Android PR
aarch64 Fedora Rawhide PR
aarch64 CentOS9 Refleaks PR
aarch64 RHEL8 PR
AMD64 Arch Linux VintageParser PR
aarch64 Fedora Stable LTO PR
aarch64 Fedora Stable Clang Installed PR
aarch64 Fedora Rawhide NoGIL PR
aarch64 RHEL8 LTO + PGO PR
AMD64 Arch Linux Asan Debug PR
aarch64 Fedora Rawhide Refleaks PR

alex · 2024-11-27T13:49:25Z

I think this looks right, but I've never actually done the process myself :-)

The only other thing we might want is bumping the version that's used in the Windows and macOS builds.

ned-deily · 2024-11-27T14:27:29Z

I believe @gpshead has done these updates most recently.

ned-deily · 2024-11-27T14:44:09Z

The only other thing we might want is bumping the version that's used in the Windows and macOS builds.

Up to now, we've treated that as a separate decision, independent of what version(s) CPython supports. In particular, we have been taking the conservative approach and sticking with OpenSSL 3.0.x which is documented as that project's LTS version. We will eventually have to revisit that, i.e. before 2026-09-07, but it doesn't need to be part of this process.

petermarko · 2024-11-27T22:10:01Z

This does not seem to fix #125936
After picking commit "Generate data for OpenSSL 3.4 and add it to multissltests" to 3.13.0, the test are still failing.
I don't see the afected tests being run in https://github.com/python/cpython/actions/runs/12051565064/job/33602924793
Not sure what I'm missing here...

encukou · 2024-11-28T08:52:30Z

@petermarko, yes, this is better error reporting in general, but doesn't affect the bug we're getting there. I'll send another PR for a better error message for that case, and then, hopefully, one to fix the underlying bug.

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e271e9cbf896f1fb97d56c426e4217a6d2105ea4) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

… process (pythonGH-127331) - Add `git describe` output to headers generated by `make_ssl_data.py` This info is more important than the date when the file was generated. It does mean that the tool now requires a Git checkout of OpenSSL, not for example a release tarball. - Regenerate the older file to add the info. To the other older file, add a note about manual edits. - Add notes on how to add a new OpenSSL version - Add 3.4 error messages and multissl tests

rossburton · 2025-04-30T10:10:56Z

Any chance of this being backported to the 3.13 branch? As can be seen in the reference list above, Yocto is carrying this patch as we build 3.13 against OpenSSL 3.4.

encukou · 2025-04-30T10:46:17Z

@Yhg1s Do you want Python 3.13 to have messages for OpenSSL 3.4's new error codes?
I see it as a feature but wouldn't mind doing the backport.

encukou · 2025-08-15T10:59:49Z

We now have PRs for OpenSSL 3.4.1 (#131618) and 3.5 (#137720). All error messages, plus improvements to the scripts that generate the files.

@Yhg1s, the question still stands. I'm happy to do the backports if you want them.

Yhg1s · 2025-08-15T13:00:19Z

Ah, sorry, I thought we'd discussed this (or maybe I just didn't make the final call). No, I don't think we should backport this to 3.13. It may seem harmless but it's the kind of thing that'll trip people up when doing patch upgrades (which we want to be as seamless as possible, evidence of the last couple of expedited releases notwithstanding.)

encukou · 2025-08-18T11:04:25Z

It's also possible I didn't record your call correctly back then.
Thanks for making it clear!

encukou added 6 commits November 26, 2024 12:50

Add notes on adding a new OpenSSL version
7ebe9aa

Generate data for OpenSSL 3.4 and add it to multissltests
16f47c6

Regenerate _ssl_data_111.h, to show the source commit
bc310fe

_ssl_data_300.h: Edit the headsr to log manual edits
2f256a3

Fix typo
e734721

encukou requested review from ezio-melotti and hugovk as code owners November 27, 2024 13:20

bedevere-appBot added the awaiting core review label Nov 27, 2024

encukou changed the title ~~gh-27330: Update for OpenSSL 3.4~~gh-127330: Update for OpenSSL 3.4Nov 27, 2024

bedevere-appBot mentioned this pull request Nov 27, 2024
Add support for OpenSSL 3.4 #127330
Closed

Add blurb
68810ef

encukou mentioned this pull request Nov 27, 2024
[tests] test_poplib fails with "env changed" on Arch Linux with OpenSSL 3.4: [SYS] unknown error (_ssl.c:2634) #127257
Closed

Use a wildcard for c-analyzer
ea72022

encukou requested a review from ericsnowcurrently as a code owner November 27, 2024 13:43

gpshead self-assigned this Nov 27, 2024

gpshead approved these changes Nov 27, 2024
View reviewed changes

Comment threadTools/ssl/make_ssl_data.py Outdated

bedevere-appBot added awaiting merge and removed awaiting core review labels Nov 27, 2024

Use UTF-8
783129a

encukou commented Nov 28, 2024
View reviewed changes

Comment threadTools/ssl/make_ssl_data.py Outdated

Missing word in comment
f4336cd

encukou merged commit db5c576 into python:mainNov 28, 2024

bedevere-appBot removed the awaiting merge label Nov 28, 2024

encukou deleted the make_ssl_data branch November 28, 2024 12:29

jcfp mentioned this pull request Dec 27, 2024
ENGINE socket.error 1 - ENGINE Unhandled error while processing an incoming connection SSLEOFError sabnzbd/sabnzbd#3006
Closed

encukou mentioned this pull request Aug 15, 2025
gh-132339: Add support for OpenSSL 3.5 #137720
Merged

This was referenced Aug 26, 2025
Add OpenSSL 3.5 support to CPython infrastructure #132339
Closed
Build with OpenSSL 3.5 astral-sh/python-build-standalone#775
Closed

Uh oh!

Uh oh!

Conversation

encukou commented Nov 27, 2024• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add git describe output to headers generated by make_ssl_data.py

Add notes on how to add a new OpenSSL version

Add 3.4 error messages and multissl tests

Uh oh!

encukou commented Nov 27, 2024

Uh oh!

bedevere-bot commented Nov 27, 2024

Uh oh!

alex commented Nov 27, 2024

Uh oh!

ned-deily commented Nov 27, 2024

Uh oh!

ned-deily commented Nov 27, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petermarko commented Nov 27, 2024

Uh oh!

Uh oh!

encukou commented Nov 28, 2024

Uh oh!

Uh oh!

rossburton commented Apr 30, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou commented Apr 30, 2025

Uh oh!

encukou commented Aug 15, 2025

Uh oh!

Yhg1s commented Aug 15, 2025

Uh oh!

encukou commented Aug 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

encukou commented Nov 27, 2024•
edited by bedevere-app Bot
Loading

Add `git describe` output to headers generated by `make_ssl_data.py`

ned-deily commented Nov 27, 2024•
edited
Loading

rossburton commented Apr 30, 2025•
edited
Loading