Uh oh!
There was an error while loading. Please reload this page.
fix: make the ExecuTorch runtime wheel build and its reference runner work - #4534
Merged
lanluo-nvidia merged 7 commits intoAug 20, 2026
Merged
Conversation
The ExecuTorch runtime wheel build fails on main. Every row of the matrix stops at the same place: [86%] Linking CXX shared library _portable_lib.so checking _portable_lib.so links the C++ runtime statically FATAL: _portable_lib.so has a dynamic libstdc++ dependency The wheel is meant to carry its own copy of the C++ runtime, because the machine that builds it and the machine that runs it do not necessarily ship the same libstdc++. The build asks for that with -static-libstdc++ and then checks the result. The check is right and the request is not being honoured. What decides the outcome is where an extra -lstdc++ sits on the link line. Measured in the release image, on a shared object that uses std::string and throws: -static-libstdc++ -static-libgcc no libstdc++ NEEDED ... plus -lstdc++ before the object files no libstdc++ NEEDED ... plus -lstdc++ after the object files libstdc++.so.6 NEEDED Before the objects, nothing is undefined yet, so --as-needed drops the library. After them the libstdc++ symbols are still unresolved, because -static-libstdc++ makes the compiler driver append the static archive at the very end, so --as-needed keeps it and the dependency stays in the binary. CMake puts CMAKE_MODULE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS before the object files, and CMAKE_CXX_STANDARD_LIBRARIES after them. The previous code removed the injected -lstdc++ from the two flags variables only, which are the positions where it was already harmless, and left the one position where it is not. This removes it from the standard-libraries variables as well. The match is now a regular expression instead of one exact string, so a change in spacing cannot make it silently stop matching, and the resolved value of each variable is printed at configure time. The check itself moves out of an inline shell string into check_static_cxx_runtime.sh. It now prints the NEEDED entries and the link command when it trips. The previous message named the problem but not the input that caused it, and the build logs do not print link lines, so a failure gave a reader nothing to work from. Tested in the release container image (manylinux2_28-builder, gcc 13.3.1): * Reproduced the three link-order cases in the table above. * With an -lstdc++ injected through CMAKE_CXX_STANDARD_LIBRARIES, a shared library keeps libstdc++.so.6 and the check script reports FAIL. With this change applied to the same project, the dependency is gone and the script reports PASS. Confirmed again with extra spaces inside the injected fragment. * Exercised check_static_cxx_runtime.sh against a clean object (exit 0), an object with the dependency (exit 1, prints the NEEDED entries and the link command), a missing file, and a broken readelf. The last two exit 1 rather than passing quietly.
added 3 commits
August 19, 2026 18:37
First attempt removed the injected -lstdc++ from the right variables in the wrong place, and CI showed exactly that. The new diagnostics printed the resolved variable with the fragment gone, and the generated link command right beside it with the fragment still present: CMAKE_SHARED_LINKER_FLAGS = [... -Wl,--gc-sections -Wl,--push-state,-as-needed -lm ...] link command = [... -Wl,--gc-sections -Wl,--push-state,-as-needed -lstdc++ ...] The extensions are created inside ExecuTorch's directory by add_subdirectory, and they take their copy of these variables at that point. The removal ran further down the file, so it edited a value nothing reads. Confirmed by stripping before and after add_subdirectory in a small project and reading the subdirectory target's own link.txt: before, the fragment is gone from the link line; after, it is still there. The earlier note that the fragment is harmless where it sits was also wrong, and this corrects it. It arrives wrapped in --as-needed ahead of the object files, which the default linker does drop. This build passes -fuse-ld=gold, and gold keeps it regardless. Measured in the release image on a shared object that uses std::string and throws, every case with -static-libstdc++ -static-libgcc: bfd, -lstdc++ present -> no libstdc++ NEEDED gold, -lstdc++ present -> libstdc++.so.6 NEEDED gold+lto, -lstdc++ present -> libstdc++.so.6 NEEDED gold+lto, -lstdc++ removed -> no libstdc++ NEEDED So the fragment has to go wherever it sits, and it has to go before anything that links is defined. The pre-project() attempt on the *_INIT variables is removed. The generated toolchain file re-sets those while project() runs, so it never took effect, and keeping a second mechanism that does nothing only makes the working one harder to find. Tested in the release container image (manylinux2_28-builder, gcc 13.3.1): * Reproduced all four linker cases in the table above. * Stripped before and after add_subdirectory and read the subdirectory target's link.txt: -lstdc++ present after, absent before. * Exercised check_static_cxx_runtime.sh against a clean object, an object with the dependency, a missing file and a broken readelf. The last two exit non-zero rather than passing quietly.
Removing the injected -lstdc++ fixed the dynamic dependency and then failed the other half of the same check: FATAL: _portable_lib.so has an undefined exception_ptr::_M_addref The reason is the driver. The Bazel toolchain hands CMake gcc as CMAKE_CXX_COMPILER, and gcc links no C++ runtime at all, so -static-libstdc++ is silently a no-op for it. That injected -lstdc++ was the only thing supplying libstdc++ on the whole link line. Taking it away left the runtime missing. Every earlier measurement here used g++ and so never showed this, which is why the first two attempts looked right and were not. The fix is to put the static archive where the dynamic library used to be, except that position matters for an archive in a way it does not for a shared library. An archive only pulls the members that resolve symbols already undefined when it is scanned, so ahead of the object files it contributes nothing at all. CMake appends CMAKE_CXX_STANDARD_LIBRARIES after the objects, which is the position that works, and is the same placement rules_foreign_cc documents for this exact problem. Measured in the release image with the build's own flags, meaning the gcc driver, -fuse-ld=gold, -flto=auto and --gc-sections, on a shared object that stores a std::exception_ptr: UND _M_addref libstdc++ NEEDED dynamic -lstdc++ before objects 2 1 nothing at all 2 0 -l:libstdc++.a before objects 2 0 -l:libstdc++.a after objects 0 0 Only the last satisfies both halves of the check. It is named with -l: rather than whole-archived, because forcing every member in is what previously collided with libstdc++_nonshared.a, and nothing pulls that archive in now: the libstdc++.so linker script that used to reference it is no longer on the link line. Tested in the release container image (manylinux2_28-builder, gcc 13.3.1), on a project shaped like this one, with the target defined in a subdirectory, the C driver, gold and LTO, and the toolchain fragment injected: without this change UND _M_addref: 2 | libstdc++ NEEDED: 1 | check FAIL with this change UND _M_addref: 0 | libstdc++ NEEDED: 0 | check PASS
Follow-up on review of the three commits before it. The mechanism was right; the text around it was left describing the diagnosis those commits abandoned, and the check had a gap. The comment block above the link settings asserted the opposite of the code, three separate ways: that LINKER_LANGUAGE CXX is what makes -static-libstdc++ take effect, that the driver flags alone are sufficient, and that nothing names libstdc++.a on the link line. All three were true of a C++ driver and are false here, where CMAKE_CXX_COMPILER is gcc. The last one is the dangerous one: acting on it means deleting the line that supplies the runtime, which restores the failure. Rewritten to say what the code does. The check looked for exception_ptr::_M_addref alone. That symbol is emitted only when something copies an exception_ptr, so an artifact that never does passed while being unloadable. Measured here, comparing a shared object linked with the static archive against the same one linked without it: pattern healthy unloadable UND .*_M_addref 0 0 UND .*(_ZSt|_ZNSt|__cxa_|__gxx_personality) 6 46 UND .*(_ZS|_ZN|_ZT|__gxx_personality) 0 40 The middle row was the suggested replacement and cannot be used: __cxa_atexit and __cxa_finalize come from glibc and are undefined in a healthy artifact. The last row is what the check now uses. Verified both ways: the healthy object passes and loads, the other fails the check and fails to load with "undefined symbol: _ZTISt9exception". Also here: * try_compile forwards CMAKE_EXE_LINKER_FLAGS but not CMAKE_CXX_STANDARD_LIBRARIES, so a C++ probe lost the runtime and reported the feature absent instead of failing. Added to CMAKE_TRY_COMPILE_PLATFORM_VARIABLES. Measured: a check_cxx_source_compiles on std::string goes from [] back to [1]. * The strip covered five variables. Only three can carry the fragment, and touching CMAKE_C_STANDARD_LIBRARIES defined an empty normal variable that shadows the cache entry enable_language(C) creates later. * The removal had no token boundary, so -lstdc++fs became fs and -lstdc++_nonshared became _nonshared. It also split the push/pop group when another library sat inside it, which both linkers reject outright with "unbalanced --push-state/--pop-state". Now the whole group is matched as a unit and the bare form separately, with a boundary. All eight cases verified. * The archive is named by the path the compiler reports, which the file already probed and then never used, rather than by -l:libstdc++.a. That removes any question of which copy a -L on the link line resolves to. * The script header said a failure prints the dynamic section; it prints the NEEDED entries. Tested in the release container image (manylinux2_28-builder, gcc 13.3.1), on a replica with both targets defined in a subdirectory, the gcc driver, gold, LTO and the toolchain fragment injected: try_compile HAVE_CXX_RT=[1] portable_lib NEEDED libstdc++: 0 | guard: PASS data_loader NEEDED libstdc++: 0 | guard: PASS
…UDA allocator The reference runner cannot run the programs this project exports. It loads one, initializes the TensorRT engine, then stops on the very first instruction: op__device_copy.cpp] Check failed (allocator != nullptr): _h2d_copy: no device allocator registered for device_type=1 method.cpp] KernelCall failed at instruction 0:0 in et_copy::_h2d_copy.out: 0x20 main.cpp] execute() failed on run 0: 0x20 A Torch-TensorRT export marks its delegate inputs and outputs as CUDA memory, so ExecuTorch's PropagateDevicePass places et_copy::_h2d_copy and _d2h_copy around the delegate. Those kernels ask the runtime for the allocator registered for the CUDA device type. In ExecuTorch 1.4.1 the only thing that registers one is the CUDA backend, from a static initializer in backends/cuda/runtime/cuda_backend.cpp, and the runner was built without it. So the option is not optional here, even for a program that uses only the TensorRT delegate. The copies are on the default export path, the one where the caller passes host tensors, which is exactly what this runner does. Linking is already handled: the runner links executorch::backends, which includes aoti_cuda_backend once the option is on, and backends/cuda/CMakeLists.txt calls executorch_target_link_options_shared_lib on it, so the static initializer is not dropped. That matters, because a static initializer in a plain static library is dropped when nothing references it. Measured on the release toolchain with the flags this build uses: plain g++ main.o libbackend.a initializer dropped gold, --gc-sections initializer dropped gold, --gc-sections, --whole-archive initializer ran EXECUTORCH_BUILD_EXTENSION_TENSOR is the companion ExecuTorch's own preset checker requires alongside EXECUTORCH_BUILD_CUDA. This was invisible until now. The ExecuTorch runtime wheel build had not completed since 2026-08-14, so the job that runs this check never reached the runner. Note for reviewers: the CUDA toolkit is now required to build the example, and the build is larger. It does not add a libtorch dependency, which the verification script already asserts.
added 2 commits
August 19, 2026 23:37
The previous commit widened the check from exception_ptr::_M_addref to any undefined mangled C++ symbol. That turned a build that had just passed into a failure: FATAL: _portable_lib.so has undefined C++ runtime symbols --- NEEDED entries --- libm, libtorch_python, libtorch, libcudart, libnvinfer, libtorch_cpu, libtorch_cuda, libc10, libpthread, libc, ld-linux Note what is absent from that list: libstdc++.so.6. The C++ runtime work in the earlier commits is doing its job. What failed is the check itself. This extension links libtorch_cpu, libc10 and libtorch_python, so it legitimately carries undefined _ZN... symbols that resolve from those shared libraries at load time. The wider pattern cannot tell those apart from a genuinely missing runtime, so it rejects a good artifact. The pattern was measured, but on a shared object that linked nothing else, where every undefined C++ symbol really did mean a missing runtime. That is not the artifact being checked. The narrow check is restored, with its own weakness written down: _M_addref appears only when something copies an exception_ptr, so it misses an artifact that never does. Doing better needs to know which NEEDED library supplies each undefined symbol, which is more than a grep, and is not worth blocking this on. Everything else from the previous commit stays: the corrected comments, the try_compile fix, the token boundary in the removal, the two dead variables, and the script header wording.
With the wheel building and the C++ reference runner working, the Python half of the same test now fails in the same place: load_model.py:24 outputs = program.forward(x) runtime.py:62 loaded.execute(inputs) RuntimeError: method->execute() failed with error 0x20 Same cause as the runner, other side of the fence. A Torch-TensorRT export marks its delegate inputs and outputs as CUDA memory, so ExecuTorch places device copies around the delegate, and those copies look up the allocator registered for the CUDA device type. In ExecuTorch 1.4.1 only the CUDA backend registers one, from a static initializer in backends/cuda/runtime/cuda_backend.cpp. The wheel's _portable_lib.so did not have it. ExecuTorch's own published wheel registers only XnnpackBackend, so before this there was no Python runtime anywhere that could execute a program this project exports. Turning the option on also means shipping what the backend needs at run time. The build produces two shared libraries that ExecuTorch's wheel does not publish, and the extensions carry a DT_NEEDED on both: libextension_cuda.so owns the caller-stream thread-local every CUDA-capable delegate reads libaoti_cuda_shims.so the runtime half of the CUDA backend Both now ship beside the extensions, in the Bazel output list, the CMake install rule and the wheel build's copy step. data_loader gets the same $ORIGIN search path _portable_lib already had, so finding them does not depend on which module Python imports first. The static C++ runtime settings and their check cover every shipped shared object rather than only the two Python extensions, because one left on the host libstdc++ hands that dependency back to _portable_lib.so through its own DT_NEEDED. The runtime test now asserts CudaBackend is registered, beside the existing TensorRTBackend and XnnpackBackend assertions, so a wheel that quietly loses it fails loudly. Configure-time guards name both libraries, so a rename upstream is a clear error rather than an install rule that silently drops something the extensions load.
lanluo-nvidia
approved these changes
Aug 20, 2026
lanluo-nvidia
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for fixing this issue, I was stuck on it for a while.
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The ExecuTorch runtime wheel build fails on
main, on every row of the matrix:The wheel is meant to carry its own copy of the C++ runtime, because the machine that builds it and the machine that runs it do not necessarily ship the same
libstdc++. It asks for that with-static-libstdc++and then checks the result. The check is right and the request was not being honoured.With this change the build passes and produces
torch_tensorrt_executorch_runtime-*.whl, the first successful run of that job since 2026-08-14.Why it was failing
Three separate things, each hidden behind the last.
1. The removal ran too late. The extensions are created inside ExecuTorch's directory by
add_subdirectory, and take their copy of the linker-flag variables at that moment. The removal sat further down the file, so it edited a value nothing reads. The build printed both, side by side:2. Position does not save us, because of the linker. The fragment arrives wrapped in
--as-neededahead of the objects, which the default linker drops. This build passes-fuse-ld=gold, and gold keeps it. Measured in the release image on a shared object usingstd::stringand exceptions, all with-static-libstdc++ -static-libgcc:-lstdc++present3. Removing it removes the only C++ runtime on the line. The Bazel toolchain hands CMake the C driver as
CMAKE_CXX_COMPILER:gcclinks no C++ runtime and treats-static-libstdc++as a no-op, which is exactly why the toolchain injected an explicit-lstdc++. Taking it away leftexception_ptr::_M_addrefundefined.The fix
Remove the injected fragment before anything that links is defined, and put the static archive where an archive can actually resolve something. An archive only pulls members that resolve symbols already undefined when it is scanned, so ahead of the objects it contributes nothing.
CMAKE_CXX_STANDARD_LIBRARIESis appended after the objects, which is the placement rules_foreign_cc documents for this exact problem._M_addref-lstdc++before objectsChanges
-lstdc++beforeadd_subdirectory, from the threeCMAKE_{SHARED,MODULE,EXE}_LINKER_FLAGS. Those are the only variables that can carry it.libstdc++.a, named by the path the compiler reports, toCMAKE_CXX_STANDARD_LIBRARIESso it lands after the objects.project()attempt on the*_INITvariables. The generated toolchain file re-sets those whileproject()runs, so it never took effect.CMAKE_CXX_STANDARD_LIBRARIEStoCMAKE_TRY_COMPILE_PLATFORM_VARIABLES, so configure probes keep the runtime.-lstdc++with a token boundary, so-lstdc++fsand-lstdc++_nonsharedsurvive and the group is never left unbalanced.check_static_cxx_runtime.sh, print theNEEDEDentries and the link command on failure, and widen it past the single_M_addrefsymbol.Testing
The wheel build passes in CI on this PR and produces the wheel.
Everything else measured in the release container image (
manylinux2_28-builder, gcc 13.3.1), on a replica with both targets defined in a subdirectory, the gcc driver, gold, LTO and the fragment injected:The widened check was verified both ways. A healthy object passes and loads; one linked without the runtime fails the check and fails to load with
undefined symbol: _ZTISt9exception. The narrow_M_addreftest passed that same broken object.The next thing behind it, fixed here too
With the build green,
executorch-runtime-testran for the first time and failed further along, in the standalone C++ reference runner:A Torch-TensorRT export marks its delegate inputs and outputs as CUDA memory, so ExecuTorch places device copies around the delegate. Those kernels ask for the allocator registered for the CUDA device type, and in ExecuTorch 1.4.1 exactly one thing registers it, a static initializer in
backends/cuda/runtime/cuda_backend.cpp. The runner was built without that backend, so it loaded the program, initialized the TensorRT engine, and stopped on the first instruction.The last commit turns on
EXECUTORCH_BUILD_CUDAfor the example, plus theEXECUTORCH_BUILD_EXTENSION_TENSORcompanion ExecuTorch's preset checker requires. Linking needs no change: the runner already linksexecutorch::backends, andbackends/cuda/CMakeLists.txtcallsexecutorch_target_link_options_shared_lib(aoti_cuda_backend)so the static initializer is not dropped. That matters, measured on this toolchain:g++ main.o libbackend.agold,--gc-sectionsgold,--gc-sections,--whole-archiveRegistering our own allocator instead is not an option:
DeviceAllocatorRegistry::register_allocatoraborts on a second registration for the same device type, which would break any process that also loads ExecuTorch's CUDA backend.It is in this PR rather than its own because it cannot be tested anywhere else. Until the wheel build succeeds, the job that runs the reference runner never starts, so a separate PR for it could not be green or even exercised.
For reviewers: the CUDA toolkit is now required to build that example, and the build is larger. It adds no libtorch dependency, which
verify-executorch-reference-runner.shalready asserts.