Skip to content

gh-133171: Re-enable JUMP_BACKWARD to free-threading build#137800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
corona10 merged 6 commits into from
Sep 24, 2025
Merged

gh-133171: Re-enable JUMP_BACKWARD to free-threading build #137800

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d470ce9
gh-133171: Re-enable JUMP_BACKWARD to free-threading build
corona10 Aug 15, 2025
cfcd9fe
Revive free-threaded build to be built.
corona10 Sep 18, 2025
644b306
Revive CI again
corona10 Sep 18, 2025
79d9226
nit
corona10 Sep 18, 2025
2aa295a
fix lint
corona10 Sep 18, 2025
fc2a93b
Merge remote-tracking branch 'upstream/main' into gh-133171-JUMP_BACK…
corona10 Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 28 additions & 28 deletions .github/workflows/jit.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -134,6 +134,34 @@ jobs:
make all --jobs 4
./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3

jit-with-disabled-gil:
name: Free-Threaded (Debug)
needs: interpreter
runs-on: ubuntu-24.04
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
llvm:
- 19
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Build with JIT enabled and GIL disabled
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
./configure --enable-experimental-jit --with-pydebug --disable-gil
make all --jobs 4
- name: Run tests
run: |
./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
continue-on-error: true

no-opt-jit:
name: JIT without optimizations (Debug)
needs: interpreter
Expand All@@ -160,31 +188,3 @@ jobs:
- name: Run tests without optimizations
run: |
PYTHON_UOPS_OPTIMIZE=0 ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3

# XXX: GH-133171
# jit-with-disabled-gil:
# name: Free-Threaded (Debug)
# needs: interpreter
# runs-on: ubuntu-24.04
# timeout-minutes: 90
# strategy:
# fail-fast: false
# matrix:
# llvm:
# - 19
# steps:
# - uses: actions/checkout@v4
# with:
# persist-credentials: false
# - uses: actions/setup-python@v5
# with:
# python-version: '3.11'
# - name: Build with JIT enabled and GIL disabled
# run: |
# sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
# export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
# ./configure --enable-experimental-jit --with-pydebug --disable-gil
# make all --jobs 4
# - name: Run tests
# run: |
# ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
6 changes: 6 additions & 0 deletions Include/internal/pycore_stackref.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -464,6 +464,12 @@ PyStackRef_CLOSE_SPECIALIZED(_PyStackRef ref, destructor destruct)
PyStackRef_CLOSE(ref);
}

static inline int
PyStackRef_RefcountOnObject(_PyStackRef ref)

@ZheaoliZheaoliSep 18, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just personal interest, correct me if I'm wrong.

It seems we don;t call this function yet right? Is this prepared for a future hook?

This function seems be trigger at

  1. PyStackRef_XCLOSE
  2. PyStackRef_CLOSE_SPECIALIZED
  3. PyStackRef_CLOSE
  4. PyStackRef_DUP

@corona10corona10Sep 18, 2025

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we don;t call this function yet right? Is this prepared for a future hook?

Yeah correct, but we can't build if this function is not added. That's all at this moment.

{
return (ref.bits & Py_TAG_REFCNT) == 0;

@colesburycolesburySep 18, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this implementation is fine, but I'm not 100% sure

}

static inline _PyStackRef
PyStackRef_DUP(_PyStackRef stackref)
{
Expand Down
5 changes: 3 additions & 2 deletions Python/bytecodes.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2940,9 +2940,10 @@ dummy_func(
};

tier1 op(_SPECIALIZE_JUMP_BACKWARD, (--)) {
#if ENABLE_SPECIALIZATION
#if ENABLE_SPECIALIZATION_FT
if (this_instr->op.code == JUMP_BACKWARD) {
this_instr->op.code = tstate->interp->jit ? JUMP_BACKWARD_JIT : JUMP_BACKWARD_NO_JIT;
uint8_t desired = tstate->interp->jit ? JUMP_BACKWARD_JIT : JUMP_BACKWARD_NO_JIT;
FT_ATOMIC_STORE_UINT8_RELAXED(this_instr->op.code, desired);
// Need to re-dispatch so the warmup counter isn't off by one:
next_instr = this_instr;
DISPATCH_SAME_OPARG();
Expand Down
5 changes: 3 additions & 2 deletions Python/generated_cases.c.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Python/optimizer.c
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -119,6 +119,7 @@ _PyOptimizer_Optimize(
PyInterpreterState *interp = _PyInterpreterState_GET();
assert(interp->jit);
assert(!interp->compiling);
#ifndef Py_GIL_DISABLED
interp->compiling = true;
// The first executor in a chain and the MAX_CHAIN_DEPTH'th executor *must*
// make progress in order to avoid infinite loops or excessively-long
Expand DownExpand Up@@ -160,6 +161,9 @@ _PyOptimizer_Optimize(
assert((*executor_ptr)->vm_data.valid);
interp->compiling = false;
return 1;
#else
return 0;
#endif
}

static _PyExecutorObject *
Expand Down
3 changes: 2 additions & 1 deletion configure
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion configure.ac
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2799,7 +2799,7 @@ AC_MSG_RESULT([$tier2_flags $jit_flags])
if test "$disable_gil" = "yes" -a "$enable_experimental_jit" != "no"; then
# GH-133171: This configuration builds the JIT but never actually uses it,
# which is surprising (and strictly worse than not building it at all):
AC_MSG_ERROR([--enable-experimental-jit cannot be used with --disable-gil.])
AC_MSG_WARN([--enable-experimental-jit does not work correctly with --disable-gil.])
fi

case "$ac_cv_cc_name" in
Expand Down
Loading