Skip to content

GH-115869: Make jit_stencils.h reproducible - #127166

Merged
brandtbucher merged 6 commits into
python:mainfrom
brandtbucher:justin-reproducible
Nov 24, 2024
Merged

GH-115869: Make jit_stencils.h reproducible#127166
brandtbucher merged 6 commits into
python:mainfrom
brandtbucher:justin-reproducible

Conversation

@brandtbucher

@brandtbucherbrandtbucher commented Nov 22, 2024

Copy link
Copy Markdown
Member

At the core dev sprint, we discussed the possibility of hosting pre-built jit_stencils.h files for common platforms and build configurations.

While there's a lot of work involved in making that happen (I'll post a rough plan shortly), this is the first logical step in that direction. This PR replaces absolute file paths in the generated code with relative ones, and processes stencils in sorted order.

@brandtbucherbrandtbucher added build The build process and cross-build 3.14 bugs and security fixes topic-JIT labels Nov 22, 2024
@brandtbucherbrandtbucher self-assigned this Nov 22, 2024
@bedevere-appbedevere-appBot mentioned this pull request Nov 22, 2024
offset = len(self.body)
padding = -offset % alignment
self.disassembly.append(f"{offset:x}: {' '.join(['00'] * padding)}")
if padding:

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unrelated cleanup.

@savannahostrowskisavannahostrowski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@brandtbucher
brandtbucher merged commit 17c16ae into python:mainNov 24, 2024
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…ythonGH-127166)
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
Keep the build tmpdir out of jit_stencils.h (pythonGH-127166)
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)
Keep the build tmpdir out of jit_stencils.h
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
@bedevere-app

Copy link
Copy Markdown

GH-154988 is a backport of this pull request to the 3.13 branch.

bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)
Keep the build tmpdir out of jit_stencils.h
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
bmwiedemann added a commit to bmwiedemann/cpython that referenced this pull request Jul 31, 2026
…7166)
Keep the build tmpdir out of jit_stencils.h
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
Yhg1s pushed a commit that referenced this pull request Aug 4, 2026
Keep the build tmpdir out of jit_stencils.h
llvm-objdump prints the path of the object file it disassembles, and the
JIT builds every stencil in a fresh tempfile.TemporaryDirectory(). That
random path ended up verbatim in the disassembly comments of the
generated jit_stencils.h, so two builds of the same source never produced
the same header:
// /tmp/tmp164olvmi/_BINARY_OP.o: file format elf64-x86-64
+// /tmp/tmpmufvewt2/_BINARY_OP.o: file format elf64-x86-64
Replace the full path with the bare file name.
Only the _parse() hunk of that commit is taken here; the rest of it
depends on refactorings that are not in 3.13.
(cherry picked from commit 17c16ae)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.14bugs and security fixesbuildThe build process and cross-buildtopic-JIT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@brandtbucher@mdboom@savannahostrowski