Crash (Sigsev) in JIT optimizer `_GUARD_LOAD_SUPER_ATTR_METHOD`

[DarkaMaul]

Bug report

Bug description:

The JIT optimizer crashes with a NULL pointer dereference (SIGSEGV at address 0x8) when tracing a polymorphic super().method() call site that has been observed with multiple different class types.

How to reproduce

Save the following as poc_stale_super_ptr.py and run with JIT stress mode:

./python.exe --version
Python 3.15.0a8+

PYTHON_JIT=0 ./python.exe poc_stale_super_ptr.py
# -> exit 0, no crash

PYTHON_JIT_STRESS=1 ./python.exe poc_stale_super_ptr.py
zsh: segmentation fault PYTHON_JIT_STRESS=1 ./python.exe poc_stale_super_ptr.py

poc_stale_super_ptr.py

Disclaimer: The PoC has been generated by an AI assistant, and I verified it worked on both my computer and a remote VM

class Base:
 def method(self):
 return 1

# 30 sibling classes each overriding method() with super().method()
class C0(Base):
 def method(self): return super().method() + 1
class C1(Base):
 def method(self): return super().method() + 1
class C2(Base):
 def method(self): return super().method() + 1
class C3(Base):
 def method(self): return super().method() + 1
class C4(Base):
 def method(self): return super().method() + 1
class C5(Base):
 def method(self): return super().method() + 1
class C6(Base):
 def method(self): return super().method() + 1
class C7(Base):
 def method(self): return super().method() + 1
class C8(Base):
 def method(self): return super().method() + 1
class C9(Base):
 def method(self): return super().method() + 1
class C10(Base):
 def method(self): return super().method() + 1
class C11(Base):
 def method(self): return super().method() + 1
class C12(Base):
 def method(self): return super().method() + 1
class C13(Base):
 def method(self): return super().method() + 1
class C14(Base):
 def method(self): return super().method() + 1
class C15(Base):
 def method(self): return super().method() + 1
class C16(Base):
 def method(self): return super().method() + 1
class C17(Base):
 def method(self): return super().method() + 1
class C18(Base):
 def method(self): return super().method() + 1
class C19(Base):
 def method(self): return super().method() + 1
class C20(Base):
 def method(self): return super().method() + 1
class C21(Base):
 def method(self): return super().method() + 1
class C22(Base):
 def method(self): return super().method() + 1
class C23(Base):
 def method(self): return super().method() + 1
class C24(Base):
 def method(self): return super().method() + 1
class C25(Base):
 def method(self): return super().method() + 1
class C26(Base):
 def method(self): return super().method() + 1
class C27(Base):
 def method(self): return super().method() + 1
class C28(Base):
 def method(self): return super().method() + 1
class C29(Base):
 def method(self): return super().method() + 1

# 5-deep inheritance chain
class D0(Base):
 def method(self): return super().method() + 1
class D1(D0):
 def method(self): return super().method() + 1
class D2(D1):
 def method(self): return super().method() + 1
class D3(D2):
 def method(self): return super().method() + 1
class D4(D3):
 def method(self): return super().method() + 1

ALL = [
 C0, C1, C2, C3, C4, C5, C6, C7, C8, C9,
 C10, C11, C12, C13, C14, C15, C16, C17, C18, C19,
 C20, C21, C22, C23, C24, C25, C26, C27, C28, C29,
 D0, D1, D2, D3, D4,
]

# Pass 1 warms specialization; pass 2 triggers JIT tracing.
for _ in range(2):
 for cls in ALL:
 cls().method()

Root cause

Commit 458aca9237 added a _GUARD_LOAD_SUPER_ATTR_METHOD optimizer case in Python/optimizer_bytecodes.

cpython/Python/optimizer_bytecodes.c

Lines 1018 to 1033 in 70eb56b

	op(_GUARD_LOAD_SUPER_ATTR_METHOD, (global_super_st, class_st, unused -- global_super_st, class_st, unused)) {
	if (sym_get_const(ctx, global_super_st) == (PyObject *)&PySuper_Type) {
	PyTypeObject *probable = (PyTypeObject *)sym_get_probable_value(class_st);
	PyTypeObject *known = (PyTypeObject *)sym_get_const(ctx, class_st);
	// not known, but has a probable type, promote the probable type
	if (known == NULL && probable != NULL && PyType_Check(probable)) {
	ADD_OP(_GUARD_NOS_TYPE_VERSION, 0, probable->tp_version_tag);
	known = probable;
	}
	sym_set_const(class_st, (PyObject *)known);
	}
	else {
	sym_set_const(global_super_st, (PyObject *)&PySuper_Type);
	sym_set_type(class_st, &PyType_Type);
	}
	}

When known == NULL but there are no probable, then sym_set_const is called with (class_st, NULL).

sym_set_const calls make_const, which then calls Py_NewRef(NULL) and finally will at some point try to increase the ref count:

cpython/Include/refcount.h

Line 298 in 70eb56b

op->ob_refcnt++;

Fix

I think that's enough to do the following check:

// Python/optimizer_bytecodes.c, line ~955
if (known != NULL) {
 sym_set_const(class_st, (PyObject *)known);
}

Found by compiling the stencils with ASAN

Stack crash

AddressSanitizer:DEADLYSIGNAL
=================================================================
==418369==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x56043dbaecb7 bp 0x7fffe30dca20 sp 0x7fffe30dca20 T0)
==418369==The signal is caused by a READ memory access.
==418369==Hint: address points to the zero page.
 #0 0x56043dbaecb7 in Py_INCREF Include/refcount.h:286
 #1 0x56043dbaed00 in _Py_NewRef Include/refcount.h:536
 #2 0x56043dbaed00 in make_const Python/optimizer_symbols.c:187
 #3 0x56043dbb23b7 in _Py_uop_sym_set_const Python/optimizer_symbols.c:632
 #4 0x56043dba448f in optimize_uops Python/optimizer_cases.c.h:2371
 #5 0x56043dbaebd4 in _Py_uop_analyze_and_optimize Python/optimizer_analysis.c:792
 #6 0x56043db958e5 in uop_optimize Python/optimizer.c:1543
 #7 0x56043db96e45 in _PyOptimizer_Optimize Python/optimizer.c:166
 #8 0x56043d95308f in stop_tracing_and_jit Python/ceval.c:1138
 #9 0x56043d99c583 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:12213
 #10 0x56043d9a03a2 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:118
 #11 0x56043d9a06ed in _PyEval_Vector Python/ceval.c:2174
 #12 0x56043d9a097c in PyEval_EvalCode Python/ceval.c:686
 #13 0x56043dbde6de in run_eval_code_obj Python/pythonrun.c:1369
 #14 0x56043dbde9e7 in run_mod Python/pythonrun.c:1472
 #15 0x56043dbdf841 in pyrun_file Python/pythonrun.c:1296
 #16 0x56043dbe2424 in _PyRun_SimpleFileObject Python/pythonrun.c:518
 #17 0x56043dbe26ad in _PyRun_AnyFileObject Python/pythonrun.c:81
 #18 0x56043dc3582e in pymain_run_file_obj Modules/main.c:411
 #19 0x56043dc35a83 in pymain_run_file Modules/main.c:430
 #20 0x56043dc3730a in pymain_run_python Modules/main.c:715
 #21 0x56043dc3794c in Py_RunMain Modules/main.c:796
 #22 0x56043dc37b0b in pymain_main Modules/main.c:826
 #23 0x56043dc37e1c in Py_BytesMain Modules/main.c:850
 #24 0x56043d56eb71 in main Programs/python.c:15
 #25 0x7f159c345ca7 (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: 58749c528985eab03e6700ebc1469fa50aa41219)
 #26 0x7f159c345d64 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29d64) (BuildId: 58749c528985eab03e6700ebc1469fa50aa41219)
 #27 0x56043d56eaa0 in _start (/home/.../python+0x326aa0) (BuildId: 0751f7a328fd60f87db90ff375974647e5b13d3a)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV Include/refcount.h:286 in Py_INCREF
==418369==ABORTING

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

[cocolato]

We need to determine why known is NULL, because if _RECORD_NOS is correct, known should not be NULL.

[cocolato]

It looks like DISPATCH_SAME_OPARG() in _SPECIALIZE_LOAD_SUPER_ATTR directly switches the same PC from a generic opcode to a specialized opcode without updating the tracing recording structure, causing _RECORD_NOS to read a null value.

[NekoAsakura]

It looks like a downstream symptom of

• JIT: _Py_uop_sym_set_recorded_type may receive a non-type at trace optimization time. #148439

cpython/Python/optimizer_symbols.c

Lines 1139 to 1145 in 54607ee

	_Py_uop_sym_set_recorded_value(JitOptContext *ctx, JitOptRef ref, PyObject *value)
	{
	// It is possible for value to be NULL due to respecialization
	// during execution of the traced instruction.
	if (value == NULL) {
	return;
	}

[cocolato]

It looks like a downstream symptom of

No, they are not the same, because in this case, _RECORD_NOS reads NULL, rather than causing an assert failure.

[cocolato]

if (ADAPTIVE_COUNTER_TRIGGERS(counter) && !IS_JIT_TRACING()) {
 next_instr = this_instr;
 _Py_Specialize_LoadSuperAttr(global_super_st, class_st, next_instr, load_method);
 DISPATCH_SAME_OPARG();
}

@Fidget-Spinner Add more check can solve this problem, but we really need to ensure that all recording operations are consistent across the family to protect re-specialization.

Refer: #148571 (comment)

[NekoAsakura]

Oh, I linked the wrong issue, meant #148571. My point was the comment in _Py_uop_sym_set_recorded_value makes it sound like NULL is expected, but downstream doesn't handle it. Either way we still need consistent recording to keep tracing.