Uh oh!
There was an error while loading. Please reload this page.
fix: unblock the ExecuTorch runtime wheel configure (Python::Python on manylinux) - #4523
Merged
lanluo-nvidia merged 1 commit intoAug 19, 2026
Conversation
shoumikhinforce-pushed
the
fix-executorch-runtime-pybind-configure
branch
3 times, most recently
from
August 19, 2026 14:37
c77fad4 to
7d01513Compare
This was referenced Aug 19, 2026
The ExecuTorch runtime wheel has never built. `executorch-runtime-build` has failed on every commit since it was added, on all ten matrix rows, and `executorch-runtime-test` is gated on it: ```yaml # .github/workflows/ci-linux-x86_64.yml executorch-runtime-test: if: ... needs.executorch-runtime-build.result == 'success' ``` That test job is the only thing that runs `tests/py/dynamo/executorch/` and the reference-runner verification, so no ExecuTorch change is covered by CI today. Because the job also fails on main, it reads as a safe pre-existing failure. ## Why it fails The build stops during CMake configure, before anything compiles: ``` Python_ADD_LIBRARY: dependent target 'Python::Python' is not defined. Did you miss to request COMPONENT 'Development.Embed'? Call Stack (most recent call first): .../pybind11/tools/pybind11NewTools.cmake:269 (python_add_library) .../executorch/codegen/tools/CMakeLists.txt:11 (pybind11_add_module) ``` ExecuTorch declares its Python extension modules as `pybind11_add_module(<target> SHARED ...)`. pybind11 maps a non-MODULE type onto `pybind11::embed`, and CMake's `python_add_library` then requires the `Python::Python` target, which exists only when Python is found with the `Development.Embed` component. Requesting that component does not work here. It needs a libpython, and the manylinux CPython has none, so the whole `find_package` fails instead: ``` Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES ...) ``` pybind11 already handles this and asks for the component optionally on purpose: ```cmake # pybind11NewTools.cmake # Development.Module support (required for manylinux) started in 3.18 set(_pybind11_dev_component Development.Module OPTIONAL_COMPONENTS Development.Embed) ``` `codegen/tools` cannot be switched off on its own. It is gated only on `EXECUTORCH_BUILD_PYBIND`, which has to stay on because this build requires `portable_lib`. ## The change Match pybind11's pattern, then fill the one gap it leaves: - `Development.Module` required, `Development.Embed` optional. Finding Python before pybind11 is the override pybind11 documents. - Where `Development.Embed` is genuinely absent, supply `Python::Python` as an empty `INTERFACE`, guarded on `NOT TARGET` so a real libpython wins when the image has one. It forwards `Python::Module` for include directories and links no libpython, which is correct for an extension module: its Python symbols resolve from the interpreter that loads it. `PYTHON_EXECUTABLE` is bridged to `Python_EXECUTABLE` first, because the build passes the interpreter under the former name while FindPython reads the latter. ExecuTorch does the same bridge but only later, so without it this call could pick a different interpreter than the rest of the build. A test pins the invariant, in the file that already pins eight properties of this same CMakeLists as text: the component must stay optional, and the stand-in must stay guarded. ## The root fix belongs upstream One keyword: `selective_build`, `portable_lib` and `data_loader` should be `MODULE`, not `SHARED`. CMake skips the check entirely for `MODULE`, no `Python::Python` is needed and no libpython is linked, which is what an extension module wants. That fixes it for every consumer. This change defers to the real target if a later pin provides one. ## Testing The configure failure only reproduces in the release image, so the mechanism was verified separately with cmake 3.30 and 4.4: - Embed absent, stand-in used: configures and links. - Embed present, real target used: configures and links. - Requesting Embed as REQUIRED on an image without libpython fails, so the mechanism is not imagined. The new test was verified in all three directions: - fixed tree: 1 passed - `Development.Embed` made required: 1 failed - stand-in removed: 1 failed Not yet confirmed on all ten matrix rows: a pull request push resolves the fast lane, which schedules one row.
shoumikhinforce-pushed
the
fix-executorch-runtime-pybind-configure
branch
from
August 19, 2026 16:11
7d01513 to
2e7552fCompareUh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The ExecuTorch runtime wheel has never built.
executorch-runtime-buildhas failedon every commit since it was added, on all ten matrix rows, and
executorch-runtime-testis gated on it:That test job is the only thing that runs
tests/py/dynamo/executorch/and thereference-runner verification, so no ExecuTorch change is covered by CI today. And
because the job also fails on main, it reads as a safe pre-existing failure.
Why it fails
The build stops during CMake configure, before anything compiles:
ExecuTorch declares its Python extension modules as
pybind11_add_module(<target> SHARED ...). pybind11 maps a non-MODULE type ontopybind11::embed, and CMake'spython_add_librarythen requiresPython::Python,which exists only when Python is found with
Development.Embed.Requesting that component does not work here. It needs a libpython, and the
manylinux CPython has none, so the whole
find_packagefails instead:pybind11 already handles this and asks for the component optionally on purpose:
codegen/toolscannot be switched off on its own. It is gated only onEXECUTORCH_BUILD_PYBIND, which has to stay on because this build requiresportable_lib.The change
Match pybind11's pattern, then fill the one gap it leaves:
Development.Modulerequired,Development.Embedoptional. Finding Python beforepybind11 is the override pybind11 documents.
Development.Embedis genuinely absent, supplyPython::Pythonas an emptyINTERFACE, guarded onNOT TARGETso a real libpython wins when the image hasone. It forwards
Python::Modulefor include directories and links no libpython,which is correct for an extension module: its Python symbols resolve from the
interpreter that loads it.
PYTHON_EXECUTABLEis bridged toPython_EXECUTABLEfirst, because the buildpasses the interpreter under the former name while FindPython reads the latter.
ExecuTorch does the same bridge but only later, so without it this call could pick a
different interpreter than the rest of the build.
A test pins the invariant, added to the file that already pins eight properties of
this same CMakeLists as text: the component must stay optional, and the stand-in
must stay guarded.
The root fix belongs upstream
One keyword:
selective_build,portable_libanddata_loadershould beMODULE,not
SHARED. CMake skips the check entirely forMODULE, noPython::Pythonisneeded and no libpython is linked, which is what an extension module wants. That
fixes it for every consumer. This change defers to the real target if a later pin
provides one.
Testing
The configure failure only reproduces in the release image, so the mechanism was
verified separately with cmake 3.30 and 4.4:
is not imagined.
The new test was verified in all three directions, because a test that cannot fail
proves nothing:
On the readelf assertion, deliberately not here
An earlier revision asserted at build time, with
readelf -d, that neitherextension links libpython. It is removed rather than kept, for three reasons:
the current ExecuTorch pin, where
strip_python_lib()already removes bothPython::Pythonandpybind11::embedfrom those targets(
executorch/CMakeLists.txt:283, called at:1161and:1195).build:
tests/py/core/test_libtorchtrt_linkage.pyrunsreadelf -don the shippedlibrary with
check=True.It is worth having as a test against the built artifact, and belongs in its own
change. Flagging it rather than dropping it quietly.
What this does not fix
With the configure fixed, the build reaches 86% and stops at a pre-existing link
failure that was previously unreachable:
That is not this change.
git diffagainst the merge base touches no line of thestatic-libstdc++ logic, and this change is purely additive. Every earlier build died
in configure, so nothing had reached the link step before. The wheel therefore needs
a second, separate fix before it builds.
Making that error visible at all required raising bazel's output limit, which is
#4524.
Also worth stating: a pull request push resolves the fast lane, so one matrix row is
scheduled rather than ten.