Data race on a shared `types.GenericAlias` iterator in `ga_iter_reduce` under free-threading (follow-up to gh-154043)

[Naserume]

Bug report

Bug description:

This is a follow-up to #154043 (segfault in ga_iternext when a types.GenericAlias iterator is shared across threads), which was fixed by gh-154108 by consuming gi->obj atomically in ga_iternext:

cpython/Objects/genericaliasobject.c

Lines 942 to 963 in 8b048eb

	ga_iternext(PyObject *op)
	{
	gaiterobject *gi = (gaiterobject*)op;
	#ifdef Py_GIL_DISABLED
	PyObject *obj = _Py_atomic_exchange_ptr(&gi->obj, NULL);
	#else
	PyObject* obj = gi->obj;
	gi->obj = NULL;
	#endif
	if (obj == NULL) {
	PyErr_SetNone(PyExc_StopIteration);
	return NULL;
	}
	gaobject *alias = (gaobject *)obj;
	PyObject *starred_alias = Py_GenericAlias(alias->origin, alias->args);
	Py_DECREF(obj);
	if (starred_alias == NULL) {
	return NULL;
	}
	((gaobject *)starred_alias)->starred = true;
	return starred_alias;
	}

That fix only covers ga_iternext. The iterator's __reduce__ implementation, ga_iter_reduce, still reads the same gi->obj field with plain (non-atomic) loads and no critical section:

cpython/Objects/genericaliasobject.c

Lines 990 to 1004 in 8b048eb

	static PyObject *
	ga_iter_reduce(PyObject *self, PyObject *Py_UNUSED(ignored))
	{
	PyObject *iter = _PyEval_GetBuiltin(&_Py_ID(iter));
	gaiterobject *gi = (gaiterobject *)self;
	/* _PyEval_GetBuiltin can invoke arbitrary code,
	* call must be before access of iterator pointers.
	* see issue #101765 */
	if (gi->obj)
	return Py_BuildValue("N(O)", iter, gi->obj);
	else
	return Py_BuildValue("N(())", iter);
	}

So sharing one iter(list[int]) across threads and calling __reduce__ concurrently with next() is still a data race on gi->obj, the same field the fix made atomic in ga_iternext. ga_iternext does _Py_atomic_exchange_ptr(&gi->obj, NULL) (atomic write) while ga_iter_reduce does if (gi->obj) return Py_BuildValue("N(O)", iter, gi->obj); (plain read).

from threading import Thread, Barrier
_shared_iter = iter(list[int])

def chain1_thread():
 for _ in range(20000):
 try:
 next(_shared_iter)
 except StopIteration:
 pass

def chain2_thread():
 for _ in range(20000):
 try:
 _shared_iter.__reduce__()
 except Exception:
 pass

N_C1 = 4
N_C2 = 4
barrier = Barrier(N_C1 + N_C2)

def _c1():
 barrier.wait()
 chain1_thread()

def _c2():
 barrier.wait()
 chain2_thread()

threads = [Thread(target=_c1) for _ in range(N_C1)]
threads += [Thread(target=_c2) for _ in range(N_C2)]
for t in threads: t.start()
for t in threads: t.join()

TSAN Report:

WARNING: ThreadSanitizer: data race (pid=650295)
 Write of size 8 at 0x7fffb6161af0 by thread T1:
 #0 _Py_atomic_exchange_ptr /home/fuzz_cpython/cpython-latest/cpython/./Include/cpython/pyatomic_gcc.h:195:10 
 #1 ga_iternext /home/fuzz_cpython/cpython-latest/cpython/Objects/genericaliasobject.c:946:21 
 #2 builtin_next /home/fuzz_cpython/cpython-latest/cpython/Python/bltinmodule.c:1776:11 
 #3 cfunction_vectorcall_FASTCALL /home/fuzz_cpython/cpython-latest/cpython/Objects/methodobject.c:449:24 
 #4 _PyObject_VectorcallTstate /home/fuzz_cpython/cpython-latest/cpython/./Include/internal/pycore_call.h:144:11 
 #5 PyObject_Vectorcall /home/fuzz_cpython/cpython-latest/cpython/Objects/call.c:327:12 
 #6 _Py_VectorCallInstrumentation_StackRefSteal /home/fuzz_cpython/cpython-latest/cpython/Python/ceval.c:768:11 
 #7 _PyEval_EvalFrameDefault /home/fuzz_cpython/cpython-latest/cpython/Python/generated_cases.c.h:1906:35
...

 Previous read of size 8 at 0x7fffb6161af0 by thread T8:
 #0 ga_iter_reduce /cpython/Objects/genericaliasobject.c:1000:13 
 #1 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:4330:35 
...

SUMMARY: ThreadSanitizer: data race /cpython/./Include/cpython/pyatomic_gcc.h:195:10 in _Py_atomic_exchange_ptr

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-154916: Fix data race in ga_iter_reduce under free-threading #154944
• gh-154916: Fix data race on a shared types.GenericAlias iterator in ga_iter_reduce under free-threading #155476
• gh-154916: Fix data races in GenericAlias using critical sections #155634
• [3.15] gh-154916: Fix data races in GenericAlias using critical sections (GH-155634) #155705

[ByteFlowing1337]

Thanks for your report! I created #155476 to fix the issue.