gh-129987: Disable GCC SLP autovectorization for the interpreter loop on x86-64

[mpage]

#131750 mysteriously caused a ~6% regression for the free-threaded build. The cause was poor code generation of opcode dispatch in the interpreter loop. Before the change the dispatch code looked like:

/root/src/cpython/Python/generated_cases.c.h:8808 [LOAD_FAST_BORROW]
DISPATCH();
19cd0a: mov -0x268(%rbp),%rsi
19cd11: movzbl %ah,%ecx
19cd14: movzbl %al,%eax
19cd17: mov %ecx,%r10d
19cd1a: jmp *(%rsi,%rax,8)

After the change, the dispatch code looked like:

# Shared dispatch code
/root/src/cpython/Python/generated_cases.c.h:81 [BINARY_OP]
DISPATCH();
19dd67: mov -0x280(%rbp),%r10
19dd6e: movzbl %ah,%ecx
19dd71: movzbl %al,%eax
19dd74: mov %ecx,%r14d
19dd77: mov -0x270(%rbp),%rcx
19dd7e: mov (%rcx,%rax,8),%rdx
19dd82: nopw 0x0(%rax,%rax,1)
19dd88: movq -0x258(%rbp),%xmm0
19dd90: movq %r12,%xmm4
19dd95: punpcklqdq %xmm4,%xmm0
19dd99: movhlps %xmm0,%xmm3
19dd9c: movq %xmm0,%r15
19dda1: movq %xmm3,%r11
19dda6: mov %r11,%rcx
19dda9: jmp *%rdx
# Duplicated dispatch code
/root/src/cpython/Python/generated_cases.c.h:8808 [LOAD_FAST_BORROW]
DISPATCH();
19dde4: movzbl %ah,%ecx
19dde7: movzbl %al,%eax
19ddea: mov %ecx,%r14d
19dded: mov -0x270(%rbp),%rcx
19ddf4: mov (%rcx,%rax,8),%rdx
19ddf8: jmp 19dd99 <_PyEval_EvalFrameDefault+0x289>

There are two problems:

1. We now have two jumps (one direct jump to the shared dispatch logic and one indirect jump to the next opcode handler) instead of one (the indirect jump to the opcode handler).
2. There's a significant amount of register shuffling in the shared dispatch code.

Both of these problems appear to be caused by GCC's SLP autovectorizer. After the change, it decides to store both the next_instr pointer and the stack_pointer in a single 128 bit register in the shared basic block that contains the opcode dispatch. This is introduced in the slp1 pass (tree dump below):

 _24061 = VIEW_CONVERT_EXPR<long unsigned int>(stack_pointer_14587);
_24062 = VIEW_CONVERT_EXPR<long unsigned int>(next_instr_14097);
_24063 = {_24062, _24061};
<bb 19> [count: 1658034300]:
# frame_2363(ab) = PHI <frame_20485(4258), frame_20519(18)>
# oparg_1245(ab) = PHI <oparg_20252(4258), oparg_14635(18)>
# next_instr_1246(ab) = PHI <next_instr_11924(4258), next_instr_14097(18)>
# stack_pointer_2976(ab) = PHI <stack_pointer_20484(4258), stack_pointer_14587(18)>
# _3209 = PHI <_20217(4258), _20681(18)>
# # Combination of next_instr and stack_pointer:
# # vect_next_instr_1246.7061_24064 = PHI <vect_next_instr_11924.7060_24060(4258), _24063(18)>
_24067 = BIT_FIELD_REF <vect_next_instr_1246.7061_24064, 64, 64>;
_24068(ab) = (union _PyStackRef *) _24067;
_24065 = BIT_FIELD_REF <vect_next_instr_1246.7061_24064, 64, 0>;
_24066(ab) = (union _Py_CODEUNIT *) _24065;
# DEBUG stack_pointer => stack_pointer_2976(ab)
# DEBUG next_instr => next_instr_1246(ab)
# DEBUG oparg => oparg_1245(ab)
# DEBUG frame => frame_2363(ab)
goto _3209;

Disabling the SLP autovectorization pass for the interpreter loop fixes both problems. After this change the opcode dispatch code looks like:

/root/src/cpython/Python/generated_cases.c.h:8808 [LOAD_FAST_BORROW]
DISPATCH();
19aa37: mov -0x260(%rbp),%rsi
19aa3e: movzbl %ah,%ecx
19aa41: movzbl %al,%eax
19aa44: movslq %ecx,%r15
19aa47: jmp *(%rsi,%rax,8)

Performance improves by ~8% for the free-threaded build.

Surprisingly, this also seems to improve performance for the default build by ~4%. I don't understand why and I don't fully trust the result. The generated dispatch code for the default build looks unaffected by this change. Additionally, measuring instructions retired using fastbench shows a negligible change, whereas it shows a ~8% reduction for the free-threaded build.

• Issue: computed-goto interpreter: Prevent the compiler from merging DISPATCH calls #129987

[pinskia]

I think this is the same as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115777 .

[colesbury]

Nice!

[cielavenir]

Seems this broke gdb py-bt: #142095