gh-146207: Add support for OpenSSL 4.0.0 alpha1

[vstinner]

OpenSSL 4.0.0 alpha1 no longer defines the symbols:

• SSLv3_method
• TLSv1_method
• TLSv1_1_method
• TLSv1_2_method

• Issue: Building with OpenSSL-4.0.0-alpha1 fails with #146207

[vstinner]

I didn't test my own change on OpenSSL 4.0.0 alpha1, I only asked @heitbaum to test my change.

[zware]

@vstinner This patch (should, probably :) ) be enough to get you a test in CI, even if we don't want to merge it yet:

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 2fa2ab768dc..40feaef01ce 100644
--- a/.github/workflows/build.yml+++ b/.github/workflows/build.yml@@ -275,6 +275,7 @@ jobs:
- { name: openssl, version: 3.4.4 }
- { name: openssl, version: 3.5.5 }
- { name: openssl, version: 3.6.1 }
+ - { name: openssl, version: 4.0.0-alpha1 }
## AWS-LC
- { name: aws-lc, version: 1.68.0 }
env:

[vstinner]

I applied @zware's patch: Tests / Ubuntu SSL tests (ubuntu-24.04, openssl, 4.0.0-alpha1) job is currently running.

@vstinner This patch (should, probably :) ) be enough to get you a test in CI, even if we don't want to merge it yet

Yeah, I will revert this patch once we get the openssl, 4.0.0-alpha1 CI job result.

[zware]

Not as simple as hoped :(. I can reproduce the CI failure locally, though; there's a missing symlink (or openssl is just looking in the wrong place) for lib -> lib64 in the openssl install directory.

With that, I get these failures:

2 tests failed: test_ssl test_urllib2_localnet
43 tests OK.
0:00:23 load avg: 3.07 Re-running 2 failed tests in verbose mode in subprocesses
0:00:23 load avg: 3.07 Run 2 tests in parallel using 2 worker processes
0:00:24 load avg: 3.07 [1/2/1] test_ssl failed (1 failure)
Re-running test_ssl in verbose mode (matching: test_openssl_version)
test_ssl: testing with 'OpenSSL 4.0.0-alpha1 10 Mar 2026' (4, 0, 0, 0, 0)
under 'Linux-6.19.7-200.fc43.x86_64-x86_64-with-glibc2.42'HAS_SNI=TrueOP_ALL=0x80000050OP_NO_TLSv1_1=0x10000000
test_openssl_version (test.test_ssl.BasicSocketTests.test_openssl_version) ... FAIL
======================================================================
FAIL: test_openssl_version (test.test_ssl.BasicSocketTests.test_openssl_version)
----------------------------------------------------------------------
Traceback (most recent call last): File "/path/to/cpython/bump_multissl_awslc/Lib/test/test_ssl.py", line 590, in test_openssl_versionself.assertLess(n, 0x40000000) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: 1073741824 not less than 1073741824 
---------------------------------------------------------------------- Ran 1 test in 0.007s FAILED (failures=1) test test_ssl failed
0:00:24 load avg: 3.07 [2/2/2] test_urllib2_localnet failed (2 errors) Re-running test_urllib2_localnet in verbose mode (matching: test_https, test_https_sni)
test_https (test.test_urllib2_localnet.TestUrlopen.test_https) ... ERROR
stopping HTTPS server joining HTTPS thread test_https_sni (test.test_urllib2_localnet.TestUrlopen.test_https_sni) ... ERROR
stopping HTTPS server joining HTTPS thread ======================================================================
ERROR: test_https (test.test_urllib2_localnet.TestUrlopen.test_https)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/cpython/bump_multissl_awslc/Lib/test/test_urllib2_localnet.py", line 569, in test_https
data =self.urlopen("https://localhost:%s/bizarre"% handler.port, context=context)
File "/path/to/cpython/bump_multissl_awslc/Lib/test/test_urllib2_localnet.py", line 473, in urlopen
l.extend(f.readlines(200))
~~~~~~~~~~~^^^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/http/client.py", line 714, in readline
result =self.fp.readline(limit)
File "/path/to/cpython/bump_multissl_awslc/Lib/socket.py", line 734, in readintoreturnself._sock.recv_into(b)
~~~~~~~~~~~~~~~~~~~~^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/ssl.py", line 1355, in recv_intoreturnself.read(nbytes, buffer)
~~~~~~~~~^^^^^^^^^^^^^^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/ssl.py", line 1154, in readreturnself._sslobj.read(len, buffer)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ssl.SSLError: A failure in the SSL library occurred (_ssl.c:2977)
======================================================================
ERROR: test_https_sni (test.test_urllib2_localnet.TestUrlopen.test_https_sni)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/cpython/bump_multissl_awslc/Lib/test/test_urllib2_localnet.py", line 585, in test_https_sniself.urlopen("https://localhost:%s"% handler.port, context=context)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/test/test_urllib2_localnet.py", line 473, in urlopen
l.extend(f.readlines(200))
~~~~~~~~~~~^^^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/http/client.py", line 714, in readline
result =self.fp.readline(limit)
File "/path/to/cpython/bump_multissl_awslc/Lib/socket.py", line 734, in readintoreturnself._sock.recv_into(b)
~~~~~~~~~~~~~~~~~~~~^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/ssl.py", line 1355, in recv_intoreturnself.read(nbytes, buffer)
~~~~~~~~~^^^^^^^^^^^^^^^^
File "/path/to/cpython/bump_multissl_awslc/Lib/ssl.py", line 1154, in readreturnself._sslobj.read(len, buffer)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ssl.SSLError: A failure in the SSL library occurred (_ssl.c:2977)
----------------------------------------------------------------------
Ran 2 tests in 0.122s
FAILED (errors=2)
test test_urllib2_localnet failed
2 tests failed again:
test_ssl test_urllib2_localnet

[vstinner]

I tested multissltests.py using:

python3 Tools/ssl/multissltests.py --steps=library --base-directory "$PWD/multissl" --openssl '4.0.0-alpha1' --system Linux
./configure --with-pydebug --with-openssl=$PWD/multissl/openssl/4.0.0-alpha1/
make

I fixed a few more issues, but make fails with OPENSSL_sk_set_thunks missing symbol:

[ERROR] _ssl failed to import: /home/vstinner/python/main/build/lib.linux-x86_64-3.15/_ssl.cpython-315d-x86_64-linux-gnu.so: undefined symbol: OPENSSL_sk_set_thunks
[ERROR] _ssl (/home/vstinner/python/main/build/lib.linux-x86_64-3.15/_ssl.cpython-315d-x86_64-linux-gnu.so) is missing

[vstinner]

there's a missing symlink (or openssl is just looking in the wrong place) for lib -> lib64 in the openssl install directory.

I fixed that with a5152be.

[heitbaum]

Tested results in #146207 (comment)

[vstinner]

@picnixz@gpshead@zware: Would you mind to review this change?

I reverted the change to run "Tests / Ubuntu SSL tests (ubuntu-24.04, openssl, 4.0.0-alpha1)" CI job. I don't think that we should test alpha versions in our CI.

There are still two test_urllib2_localnet failures that I failed to fix. I cannot test OpenSSL 4 locally (on Fedora), I get an error about the OPENSSL_sk_set_thunks symbol (the CI wasn't impacted by this issue). IMO this PR is good enough for a first step to support OpenSSL 4.

Also, I think that we should backport the change to Python 3.13 and 3.14 to also prepare these branches to OpenSSL 4.

[picnixz]

I am not very fond of adding support in stable branches for an alpha version for now as I do not know if/how the ABI could change.

I do not mind having a presupport in 3.15 so that we can get feedback during our beta but I will take a look at what is planned in OpenSSL once I am at home.

[miss-islington-app]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

[bedevere-app]

GH-146403 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-146404 is a backport of this pull request to the 3.13 branch.

[vstinner]

I merged my PR. Thanks for reviews.

@picnixz:

I am not very fond of adding support in stable branches for an alpha version for now as I do not know if/how the ABI could change.

Oops, I forgot to remove "backport" labels. I closed the 3.13 and 3.14 backport PRs (without merging them).

[vstinner]

Oh, I understood the OPENSSL_sk_set_thunks symbol error: make clean removes all .so files and so removes libcrypto.so and libssl.so :-D The solution is to not run make clean or to create multissl/ directory outside CPython source tree. The workaround is to recreate the symlinks manually:

cd multissl/openssl/4.0.0-alpha1/lib64
ln -s libcrypto.so.4 libcrypto.so
ln -s libssl.so.4 libssl.so

[hugovk]

See also #148292 for ssl.SSLError: A failure in the SSL library occurred with 4.0.0-beta1.

[thesamesam]

@vstinner@picnixz Could we reopen the backport PRs? 4.0.0 final is out now. I have to admit I wasn't expecting it so soon either..

[vstinner]

@vstinner@picnixz Could we reopen the backport PRs? 4.0.0 final is out now. I have to admit I wasn't expecting it so soon either.

I created #148600 for OpenSSL 4.0.0 final. I'm also surprised with the short delay between the alpha1 and the final release (1 month and 4 days).

[picnixz]

I don't want to backport anything until we fixed the tests. I also don't want to backport anything until we have more feedback from the 3.15's beta. We can always backport OSSL 4.x support later for 3.13 and 3.14.