gh-140936: Fix JIT assertion crash at finalization if some generator is alive by efimov-mikhail · Pull Request #140969 · python/cpython

efimov-mikhail · 2025-11-03T22:42:24Z

I've removed setting interp->jit to false at finalization.
Because in rare case it may be a problem when some optimized code is used after finalize_modules.

New test needs subprocess since we have assertion crash only at the finalization stage.
On release builds there was no real problem, and test passed with this fix and without it.

Issue: Assertion failure in Python/optimizer.c _PyOptimizer_Optimize in JIT #140936

Fidget-Spinner · 2025-11-03T22:48:38Z

Hmm I think this isn't the problem.

The problem might be that JUMP_BACKWARD is specialized to JUMP_BACKWARD_JIT.

Following that, finalization happens, turning the JIT off. However, some of the JUMP_BACKWARD still remains as JUMP_BACKWARD_JIT.

This causes optimizer to be triggered on JUMP_BACKWARD_JIT.

The real fix should be to check if interp->jit is false in _PyOptimizer_Optimize and immediately return if not.

efimov-mikhail · 2025-11-03T22:56:17Z

CC @brandtbucher as author of #129194

efimov-mikhail · 2025-11-03T23:08:00Z

The real fix should be to check if interp->jit is false in _PyOptimizer_Optimize and immediately return if not.

Ok, I've provided this fix.

Fidget-Spinner · 2025-11-03T23:11:38Z

@efimov-mikhail thanks. Just checking: does it pass the test on your system? I didn't test it but I think it should.

efimov-mikhail · 2025-11-03T23:20:36Z

On my system new test passes with this fix and fails without it on debug build.
And all others tests also pass.

sergey-miryanov

LGTM

efimov-mikhail · 2025-11-12T12:47:21Z

Friendly ping, @Fidget-Spinner, @ZeroIntensity .
Can we merge this or there's something else that I have to address?

miss-islington-app · 2025-11-12T19:04:06Z

Thanks @efimov-mikhail for the PR, and @Fidget-Spinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington-app · 2025-11-12T19:04:09Z

Sorry, @efimov-mikhail and @Fidget-Spinner, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 35ed3e4cedc8aef3936da81a6b64e90374532b13 3.14

efimov-mikhail · 2025-11-12T19:22:43Z

Thanks for the merge, @Fidget-Spinner. I'll provide manual backport for 3.14.

…rator is alive (pythonGH-140969)

bedevere-app · 2025-11-13T07:25:15Z

GH-141494 is a backport of this pull request to the 3.14 branch.

…erator is alive (GH-140969) (GH-141494) gh-140936: Fix JIT assertion crash at finalization if some generator is alive (GH-140969)

* main: (463 commits) pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603) pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974) pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310) pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521) pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884) pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668) pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378) pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009) pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496) pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128) pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499) pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652) pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909) pythongh-137109: refactor warning about threads when forking (python#141438) pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405) pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103) pythongh-141442: Add escaping to iOS testbed arguments (python#141443) pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969) Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167) pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428) ...

* 'main' of github.com:python/cpython: (464 commits) pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603) pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974) pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310) pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521) pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884) pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668) pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378) pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009) pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496) pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128) pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499) pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652) pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909) pythongh-137109: refactor warning about threads when forking (python#141438) pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405) pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103) pythongh-141442: Add escaping to iOS testbed arguments (python#141443) pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969) Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167) pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428) ...

…rator is alive (pythonGH-140969)

efimov-mikhail added 2 commits November 4, 2025 01:03

Possible solution
cc1f1e9

Test added
937b990

efimov-mikhail requested review from ZeroIntensity and ericsnowcurrently as code owners November 3, 2025 22:42

bedevere-appBot added the awaiting review label Nov 3, 2025

bedevere-appBot mentioned this pull request Nov 3, 2025
Assertion failure in Python/optimizer.c _PyOptimizer_Optimize in JIT #140936
Closed

efimov-mikhail requested review from Fidget-Spinner and sergey-miryanov November 3, 2025 22:42

efimov-mikhail added skip news needs backport to 3.14 bugs and security fixes labels Nov 3, 2025

efimov-mikhail requested a review from markshannon as a code owner November 3, 2025 23:07

Another solution
688f063

Fidget-Spinner reviewed Nov 3, 2025
View reviewed changes

Comment threadLib/test/test_capi/test_opt.py Outdated

New line fix in test
39f9bc3

Fidget-Spinner approved these changes Nov 4, 2025
View reviewed changes

bedevere-appBot added awaiting merge and removed awaiting review labels Nov 4, 2025

ZeroIntensity reviewed Nov 4, 2025
View reviewed changes

Comment threadLib/test/test_capi/test_opt.py Outdated

efimov-mikhail requested a review from ZeroIntensity November 4, 2025 08:17

Simplify test
28c07d6

sergey-miryanov reviewed Nov 4, 2025
View reviewed changes

Comment threadLib/test/test_capi/test_opt.py

sergey-miryanov approved these changes Nov 4, 2025
View reviewed changes

Fidget-Spinner reviewed Nov 6, 2025
View reviewed changes

Comment threadLib/test/test_capi/test_opt.py Outdated

Test improvement
e7a261a

Fidget-Spinner reviewed Nov 12, 2025
View reviewed changes

Comment threadPython/optimizer.c

efimov-mikhail added 2 commits November 12, 2025 16:57

Merge branch 'main' into issue-140936-jit-crash-at-finalization
21a6e2b

Some comment at _PyOptimizer_Optimize
6be7a84

Fidget-Spinner merged commit 35ed3e4 into python:mainNov 12, 2025
62 checks passed

bedevere-appBot removed the awaiting merge label Nov 12, 2025

miss-islington-appBot assigned Fidget-Spinner Nov 12, 2025

efimov-mikhail added a commit to efimov-mikhail/cpython that referenced this pull request Nov 13, 2025
pythongh-140936: Fix JIT assertion crash at finalization if some gene…
e996564
…rator is alive (pythonGH-140969)

bedevere-appBot removed the needs backport to 3.14 bugs and security fixes label Nov 13, 2025

Fidget-Spinner pushed a commit that referenced this pull request Nov 13, 2025
[3.14]gh-140936: Fix JIT assertion crash at finalization if some gen…
48b7d75
…erator is alive (GH-140969) (GH-141494) gh-140936: Fix JIT assertion crash at finalization if some generator is alive (GH-140969)

StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
pythongh-140936: Fix JIT assertion crash at finalization if some gene…
3ff68f7
…rator is alive (pythonGH-140969)

Uh oh!

Uh oh!

Conversation

efimov-mikhail commented Nov 3, 2025• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fidget-Spinner commented Nov 3, 2025

Uh oh!

efimov-mikhail commented Nov 3, 2025

Uh oh!

efimov-mikhail commented Nov 3, 2025

Uh oh!

Fidget-Spinner commented Nov 3, 2025

Uh oh!

efimov-mikhail commented Nov 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergey-miryanov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

efimov-mikhail commented Nov 12, 2025

Uh oh!

Uh oh!

Uh oh!

miss-islington-appBot commented Nov 12, 2025

Uh oh!

miss-islington-appBot commented Nov 12, 2025

Uh oh!

efimov-mikhail commented Nov 12, 2025

Uh oh!

bedevere-appBot commented Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

efimov-mikhail commented Nov 3, 2025•
edited by bedevere-app Bot
Loading