Skip to content

GH-118943: Fix another race condition when generating jit_stencils.h - #120690

Merged
brandtbucher merged 2 commits into
python:mainfrom
hroncok:jitmakerace
Aug 5, 2024
Merged

GH-118943: Fix another race condition when generating jit_stencils.h#120690
brandtbucher merged 2 commits into
python:mainfrom
hroncok:jitmakerace

Conversation

@hroncok

@hroncokhroncok commented Jun 18, 2024•
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Contributor

Another process might have already moved jit_stencils.h.new

@hroncok
pythonGH-118943: Fix another race condition when generating jit_stenc…
60b01e7
…ils.h
Another process might have already moved jit_stencils.h.new
@bedevere-appbedevere-appBot mentioned this pull request Jun 18, 2024
Closed
Eclips4
Eclips4 reviewed Jun 18, 2024
View reviewed changes
Comment threadMisc/NEWS.d/next/Build/2024-06-18-15-28-25.gh-issue-118943.aie7nn.rst Outdated
@hroncok@Eclips4
Update Misc/NEWS.d/next/Build/2024-06-18-15-28-25.gh-issue-118943.aie…
e4edbc5
…7nn.rst
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
@brandtbucher

brandtbucher commented Jun 18, 2024

Copy link
Copy Markdown
Member

Thanks for this, it seems like a nice fix for the issue.

More generally though, I'm surprised that two jit_stencils.h files are being written in parallel like this for the same build. That shouldn't be happening, normally.

This seems to be the source of all of the issues you've been finding. Our Makefile is set up to only run one of these jobs per make target... I'm not sure that running several targets (like regen-all and all) in parallel is really intended. Is that what your build does?

@brandtbucherbrandtbucher added build The build process and cross-build needs backport to 3.13 bugs and security fixes topic-JIT labels Jun 18, 2024
@hroncok

hroncok commented Jun 18, 2024

Copy link
Copy Markdown
ContributorAuthor

We run regen-all first, then all. Are you interested in full logs?

@brandtbucher

brandtbucher commented Jun 18, 2024

Copy link
Copy Markdown
Member

Ah, I think I see the issue! regen-jit is part of regen-all, but regen-all already builds the JIT for other reasons.

I think we should remove regen-jit from the regen-all target. It's really intended for files that will be checked in and aren't generated as part of the normal build.

@hroncok

hroncok commented Jun 28, 2024

Copy link
Copy Markdown
ContributorAuthor

This was not merged in time for 3.13.0b3. How can I move this forward?

@hroncok

hroncok commented Jul 9, 2024

Copy link
Copy Markdown
ContributorAuthor

@brandtbucher Could you please merge this? Is this waiting for something else?

@hroncok

hroncok commented Aug 1, 2024

Copy link
Copy Markdown
ContributorAuthor

@Yhg1s Hey Thomas. We have carried this patch in Fedora since 3.13.0b2. Is there any chance we could get this merged to at least 3.13.0rc2?

@Eclips4

Eclips4 commented Aug 1, 2024•
edited
Loading

Copy link
Copy Markdown
Member

@Yhg1s Hey Thomas. We have carried this patch in Fedora since 3.13.0b2. Is there any chance we could get this merged to at least 3.13.0rc2?

I've pinged Brandt in the private coredev chat on Discord to get his attention.

brandtbucher
brandtbucher approved these changes Aug 1, 2024
View reviewed changes

@brandtbucherbrandtbucher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for letting this slip! Looks good to me. One thing, though: can you also remove regen-jit from the regen-all target in Makefile.pre.in?

@Yhg1s I think this is reasonable to backport to the next RC. It's just minor fixes for race conditions in the JIT build (thanks @hroncok for your work to make it available on Fedora).

@brandtbucherbrandtbucher self-assigned this Aug 1, 2024
@Yhg1s

Yhg1s commented Aug 1, 2024

Copy link
Copy Markdown
Member

Sure, this is fine to get in 3.13.0rc2.

@hroncok

hroncok commented Aug 2, 2024

Copy link
Copy Markdown
ContributorAuthor

One thing, though: can you also remove regen-jit from the regen-all target in Makefile.pre.in?

I'm happy to do that in a separate PR.

@hroncok

hroncok commented Aug 2, 2024

Copy link
Copy Markdown
ContributorAuthor

See #122602

@erlend-aaslanderlend-aasland mentioned this pull request Aug 2, 2024
Merged
@hroncok

hroncok commented Aug 5, 2024

Copy link
Copy Markdown
ContributorAuthor

Thanks for the second approval @brandtbucher. Could you please merge this as well?

@brandtbucher
brandtbucher merged commit 44659d3 into python:mainAug 5, 2024
@miss-islington-app

miss-islington-appBot commented Aug 5, 2024

Copy link
Copy Markdown

Thanks @hroncok for the PR, and @brandtbucher for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 5, 2024
@hroncok@Eclips4@miss-islington
pythonGH-118943: Handle races when moving jit_stencils.h (pythonGH-12…
0514a52
…0690)
(cherry picked from commit 44659d3)
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
@bedevere-app

bedevere-appBot commented Aug 5, 2024

Copy link
Copy Markdown

GH-122709 is a backport of this pull request to the 3.13 branch.

@bedevere-appbedevere-appBot removed the needs backport to 3.13 bugs and security fixes label Aug 5, 2024
@hroncok
hroncok deleted the jitmakerace branch August 5, 2024 23:20
@hroncok

hroncok commented Aug 5, 2024

Copy link
Copy Markdown
ContributorAuthor

Thanks.

brandtbucher pushed a commit to brandtbucher/cpython that referenced this pull request Aug 7, 2024
@hroncok@Eclips4@brandtbucher
pythonGH-118943: Handle races when moving jit_stencils.h (pythonGH-12…
84d0b2c
…0690)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
This was referenced Aug 8, 2024
Merged
Merged
blhsing pushed a commit to blhsing/cpython that referenced this pull request Aug 22, 2024
@hroncok@Eclips4@blhsing
pythonGH-118943: Handle races when moving jit_stencils.h (pythonGH-12…
0ce8363
…0690)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Eclips4Eclips4Eclips4 left review comments
@brandtbucherbrandtbucherbrandtbucher approved these changes

Assignees

@brandtbucherbrandtbucher

Labels

buildThe build process and cross-buildtopic-JIT

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hroncok@brandtbucher@Eclips4@Yhg1s