gh-133171: Re-enable JUMP_BACKWARD to free-threading build

[corona10]

• Issue: Make the JIT thread-safe #133171

[corona10]

@savannahostrowski@diegorusso@Fidget-Spinner@brandtbucher

To the JIT team — I’m not sure who’s currently working on this, but it might be a good starting point to add JIT support. My next goal will be to make the tier-two instructions thread-safe.

[Fidget-Spinner]

I think we should start by running the tests in CI for JIT+FT current build and see what passes/fails before doing this

[corona10]

I think we should start by running the tests in CI for JIT+FT current build and see what passes/fails before doing this

We disabled FT + JIT build from #133179 do you think that we have to listing the current status?
Could we separate this task into making the tier 2 interpreter thread-safe (with optimizer disabled) and the optimizer thread-safe?

[Fidget-Spinner]

Could we separate this task into making the tier 2 interpreter thread-safe (with optimizer disabled) and the optimizer thread-safe?

This makes sense but actually the default build's JIT can't even run with the optimizer turned off! There's some segfaults when we turn off the optimizer lol.

[corona10]

This makes sense but actually the default build's JIT can't even run with the optimizer turned off! There's some segfaults when we turn off the optimizer lol.

How can I trigger this?

[Fidget-Spinner]

This makes sense but actually the default build's JIT can't even run with the optimizer turned off! There's some segfaults when we turn off the optimizer lol.

How can I trigger this?

Run the test suite with PYTHON_UOPS_OPTIMIZE=0 env var.

[corona10]

Run the test suite with PYTHON_UOPS_OPTIMIZE=0 env var.

Just for the record at default build:
PYTHON_UOPS_OPTIMIZE=0 ./python.exe -m test -j8

39 tests failed:
test.test_asyncio.test_pep492
test.test_multiprocessing_forkserver.test_processes
test.test_multiprocessing_spawn.test_processes test_array test_bdb
test_buffer test_code test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw
test_collections test_complex test_datetime test_embed test_float
test_htmlparser test_idle test_import test_json test_long
test_math test_patma test_peepholer test_pickle test_pow
test_random test_scope test_slice test_ssl test_struct
test_sys_settrace test_tarfile test_threading test_time test_trace
test_zipimport_support test_zoneinfo

[corona10]

Ah never mind, it need to be transformed tier 2 code first.

[corona10]

@Fidget-Spinner@brandtbucher I noticed that JIT x FT still doesn’t build even with the entire optimizer path disabled, so fix it temporarily. I’ve added a direct fallback to Tier 1; it’s slow, but fine for now. Let’s re-enable _PyOptimizer_Optimize step by step. To do that, we also need to re-enable CI (just for build testing)

[corona10]

cc @savannahostrowski Once this PR is merged, others can start collaborating on the full FT x JIT work separately.

[colesbury]

I think this implementation is fine, but I'm not 100% sure

[corona10]

If there are no major objections to this PR, I am going to merge this PR (maybe next week)

[Zheaoli]

Just personal interest, correct me if I'm wrong.

It seems we don;t call this function yet right? Is this prepared for a future hook?

This function seems be trigger at

1. PyStackRef_XCLOSE
2. PyStackRef_CLOSE_SPECIALIZED
3. PyStackRef_CLOSE
4. PyStackRef_DUP

[corona10]

It seems we don;t call this function yet right? Is this prepared for a future hook?

Yeah correct, but we can't build if this function is not added. That's all at this moment.