Skip to content

gh-125498: Update JIT builds to use LLVM 19 and use preserve_none - #125499

Merged
brandtbucher merged 48 commits into
python:mainfrom
savannahostrowski:remove-ghccc
Oct 30, 2024
Merged

gh-125498: Update JIT builds to use LLVM 19 and use preserve_none#125499
brandtbucher merged 48 commits into
python:mainfrom
savannahostrowski:remove-ghccc

Conversation

@savannahostrowski

@savannahostrowskisavannahostrowski commented Oct 15, 2024•
edited
Loading

Copy link
Copy Markdown
Member

@bedevere-appbedevere-appBot mentioned this pull request Oct 15, 2024
Closed
@savannahostrowski

savannahostrowski commented Oct 22, 2024

Copy link
Copy Markdown
MemberAuthor

So it looks like the Python version installed with brew is failing transitively...doing a bit of spelunking here.

@savannahostrowski

savannahostrowski commented Oct 22, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Okay, after a bit of digging here, my cursory diagnosis around the failing x86 macOS runs is essentially this (and I am by no means a Homebrew or Ruby expert):
- The default Python version of Homebrew was recently updated to 3.13 (see Homebrew/homebrew-core#182840). As part of this PR, the formula for Python 3.13 seem to vastly differ from 3.12...and 3.12 was also updated.
It seems like Homebrew is no longer handling overwriting files in the way it once did. More specifically, the combo of the setup-python step and then brew installing Python (via LLVM) is not cooperating. I did some testing here, and it looks like this issue only happens with 3.12 onward (preliminary run of https://github.com/savannahostrowski/test_repo_python_install/actions/runs/11453722971 but waiting on the fail-fast: false run for the full output).

Update as of 10/22:
I got a reply from a Homebrew maintainer: Homebrew/homebrew-core#195288. It looks like this is a known GHA issue that has existed since at least May and started popping up recently as a result of some updates that GitHub made to runner images. The workaround is to remove the symlink to the pre-installed Python so that Homebrew Python is used instead. They may also start investigating overwriting the symlinks when in GHA context on their end because this seems to be a prevalent issue.

@savannahostrowskisavannahostrowski mentioned this pull request Oct 23, 2024
Closed
@brandtbucherbrandtbucher mentioned this pull request Oct 24, 2024
Closed
@savannahostrowskisavannahostrowski mentioned this pull request Oct 27, 2024
Merged
brandtbucher
brandtbucher reviewed Oct 28, 2024
View reviewed changes

@brandtbucherbrandtbucher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few opportunities to tidy things up:

Comment threadTools/jit/_targets.py
Comment threadInclude/internal/pycore_jit.h
Comment threadTools/jit/jit.h Outdated
Comment threadTools/jit/jit.h Outdated
Comment threadMisc/NEWS.d/next/Core_and_Builtins/2024-10-22-04-18-53.gh-issue-125498.cFjPIn.rst Outdated
savannahostrowskiand others added 3 commits October 29, 2024 15:01
@savannahostrowski@brandtbucher
Update Misc/NEWS.d/next/Core_and_Builtins/2024-10-22-04-18-53.gh-issu…
bb1e650
…e-125498.cFjPIn.rst
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
@savannahostrowski@brandtbucher
Apply suggestions from code review
5f3ec52
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
@savannahostrowski
Address PR comments
46cee93
@savannahostrowski

savannahostrowski commented Oct 29, 2024

Copy link
Copy Markdown
MemberAuthor

Oooo, thanks for the review @brandtbucher. Great catches for cleaning things up a bit. Much simpler.

@savannahostrowskisavannahostrowski changed the title gh-125498: Replace ghccc with preserve_none in JIT buildsgh-125498: Update JIT builds to use LLVM 19 and use preserve_noneOct 30, 2024
brandtbucher
brandtbucher approved these changes Oct 30, 2024
View reviewed changes

@brandtbucherbrandtbucher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! @mdboom, I'm landing this, so we should probably also update the benchmark runners that need it. Let me know if you want help.

@brandtbucher
brandtbucher merged commit c29bbe2 into python:mainOct 30, 2024
@mdboom

mdboom commented Oct 30, 2024

Copy link
Copy Markdown
Contributor

Awesome! @mdboom, I'm landing this, so we should probably also update the benchmark runners that need it. Let me know if you want help.

Yep -- I'll do that now.

Eclips4 added a commit to mpage/cpython that referenced this pull request Nov 1, 2024
@Eclips4
Revert "pythonGH-125498: Update JIT builds to use LLVM 19 and preserv…
52e7fa3
…e_none (pythonGH-125499)"
This reverts commit c29bbe2.
@yselkowitzyselkowitz mentioned this pull request Nov 22, 2024
Closed
picnixz pushed a commit to picnixz/cpython that referenced this pull request Dec 8, 2024
@savannahostrowski@picnixz
pythonGH-125498: Update JIT builds to use LLVM 19 and preserve_none (p…
b6c667a
…ythonGH-125499)
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
@savannahostrowski@ebonnal
pythonGH-125498: Update JIT builds to use LLVM 19 and preserve_none (p…
6e0df0f
…ythonGH-125499)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brandtbucherbrandtbucherbrandtbucher approved these changes
@markshannonmarkshannonAwaiting requested review from markshannon

Assignees

No one assigned

Labels

topic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@savannahostrowski@mdboom@brandtbucher