Data race in test_ssl.test_sni_callback_race

[colesbury]

Bug report

There is a sporadic thread sanitizer reported data race in a newly added test

• gh-149816: Fix SNI callback callable race #150018

For example:

• https://github.com/python/cpython/actions/runs/26217633343/job/77144045276

Here is a summary from Claude with my edits:

---

On the free-threaded TSAN build, test.test_ssl.ContextTests.test_sni_callback_race sporadically reports a data race inside OpenSSL: one thread reads an ASN.1 string via ASN1_STRING_cmp (no lock held) while another writes the same heap block via ASN1_STRING_set (holding an internal CRYPTO_THREAD_lock rwlock). The test itself does not crash.

Reproducer

CC=clang-20 ./configure -C --disable-gil --with-pydebug --with-thread-sanitizer
make -j

for i in $(seq 1 30); do
 TSAN_OPTIONS="halt_on_error=1" \
 ./python -m test test_ssl -v -m test_sni_callback_race > /tmp/run_$i.log 2>&1
 [ $? -ne 0 ] && echo "FAILED on run $i" && break
done

Reproduces within ~15 runs on a 22-core machine.

TSAN report (abridged)

WARNING: ThreadSanitizer: data race

 Read of size 8 by thread T11:
 #0 memcmp
 #1 ASN1_STRING_cmp (libcrypto.so.3)
 ...
 #22 thread_run Modules/_threadmodule.c:388

 Previous write of size 8 by thread T12 (mutexes: write M0):
 #0 memcpy
 #1 ASN1_STRING_set (libcrypto.so.3)
 ...
 #22 thread_run Modules/_threadmodule.c:388

 Location is heap block of size 21 allocated by ASN1_STRING_set
 Mutex M0 created by CRYPTO_THREAD_lock_new (libcrypto.so.3)

SUMMARY: ThreadSanitizer: data race in memcmp

The writer holds an OpenSSL-owned rwlock; the reader does not take the same lock. Both call sites enter from two Python worker threads doing concurrent SSL handshakes on the same SSLContext.

Environment

• CPython main @ c35b0f2 (3.16.0a0, free-threading debug TSAN)
• Clang 20.1.8
• OpenSSL 3.0.13 (30 Jan 2024)
• Linux 6.8.0-101 x86_64

cc @kiri11 @encukou

Linked PRs

• gh-150191: Avoid data race in test_sni_callback_race #150193
• [3.15] gh-150191: Avoid data race in test_sni_callback_race (GH-150193) #153269
• [3.14] gh-150191: Avoid data race in test_sni_callback_race (GH-150193) #153270
• gh-150191: temporarily skip two openssl tests which have internal data races under TSAN #153320
• [3.14] gh-150191: temporarily skip two openssl tests which have internal data races under TSAN (GH-153320) #153349
• [3.15] gh-150191: temporarily skip two openssl tests which have internal data races under TSAN (GH-153320) #153350

[vstinner]

Oh, I created a duplicate issue (gh-151277), I forgot to check if the issue was already reported.