Uh oh!
There was an error while loading. Please reload this page.
Track the ExecuTorch pin and move it to new nightlies automatically - #4566
Open
shoumikhin wants to merge 15 commits into
Open
Track the ExecuTorch pin and move it to new nightlies automatically#4566shoumikhin wants to merge 15 commits into
shoumikhin wants to merge 15 commits into
Conversation
shoumikhin
commented
Aug 23, 2026
ContributorAuthor
Follow-up that depends on this: #4567 (ships only the TensorRT delegate in the runtime wheel). That change needs the prebuilt ExecuTorch runtime this pin brings in, so it should land after this one. |
shoumikhinforce-pushed
the
executorch-pin-dev-line
branch
2 times, most recently
from
August 23, 2026 18:07
2f5c384 to
7666e2aCompareshoumikhin
marked this pull request as ready for review
August 23, 2026 19:10
shoumikhinforce-pushed
the
executorch-pin-dev-line
branch
from
August 23, 2026 19:30
598939b to
1eb0a46Compareshoumikhin
marked this pull request as draft
August 23, 2026 19:31
shoumikhin
marked this pull request as ready for review
August 23, 2026 19:57
shoumikhinforce-pushed
the
executorch-pin-dev-line
branch
2 times, most recently
from
August 24, 2026 00:05
f05e654 to
a3a0360Compareshoumikhinforce-pushed
the
executorch-pin-dev-line
branch
6 times, most recently
from
August 25, 2026 22:43
19b80c9 to
d8b7e43Compareshoumikhinforce-pushed
the
executorch-pin-dev-line
branch
5 times, most recently
from
August 27, 2026 14:34
cc11a6d to
0e6e40bCompareshoumikhinforce-pushed
the
executorch-pin-dev-line
branch
4 times, most recently
from
August 28, 2026 17:19
93d26d1 to
aacc2ddCompare…nce runner gate The ExecuTorch gate exports one program, x + 1, and TensorRT takes it whole. So nothing in CI ever runs a program where TensorRT and ExecuTorch's own CUDA backend each own part of the same graph. That coalesced case is the whole point of combining the two backends, and it is not covered end to end today. There is a composition test that checks both delegates land in the file, but it never loads or runs the program. This adds the missing run. A new example, examples/torchtrt_executorch_example/export_coalesced.py, exports cos(erfinv(tanh(x))). TensorRT has no converter for erfinv, so a CudaPartitioner catch-all gives that operator to the CUDA backend while TensorRT keeps the rest. The script fails if the saved .pte does not carry both a TensorRTBackend and a CudaBackend delegate, so a partitioning change cannot quietly turn this into a TensorRT-only run that still passes. The script also writes <model>.expected next to the .pte, holding the output shape and the eager reference value for an all-ones input. Both reference runners fill inputs with 1.0 and this model is elementwise, so one number describes the whole expected output. Reading it from a file, instead of hard-coding a number in the shell script, keeps the expectation tied to the model. verify-executorch-reference-runner.sh now takes an optional third argument, the coalesced .pte. When given, it runs both the CMake-built runner and the packaged runner on it and compares every printed value against that reference. TensorRT, AOTInductor and eager PyTorch use different kernels for the same math, so the comparison uses a tolerance of 0.001 rather than matching printed digits. The existing x + 1 assertions keep the same strength. They now go through the same helper with a zero tolerance, because x + 1 on ones is exact in float32. Usage: python examples/torchtrt_executorch_example/export_coalesced.py \ --model_path=coalesced.pte .github/scripts/verify-executorch-reference-runner.sh \ model.pte kv_cache_decode.pte coalesced.pte Test plan On a Linux x86_64 host with an NVIDIA A100 GPU: - Ran export_coalesced.py. It reported delegates ['TensorRTBackend', 'CudaBackend', 'TensorRTBackend'] and wrote "[64,64]" and "0.6722" into the .expected file. - Ran the resulting .pte through the reference runner. It printed "output[0] shape=[64,64]" and first 8 values of 0.6722, an exact match to the eager result. - Deleted the aoti_cuda_blob.ptd that the CUDA backend writes and ran again. Same output, so this model needs no external weight file. - Exercised the new shell assertion helper against captured runner output: correct output passes; one wrong value fails; a wrong shape fails; a missing values line fails; a value inside the tolerance passes and one outside it fails. - shellcheck, bash -n, black and isort are clean on the changed files. Not yet observed in CI: the ExecuTorch runtime build job currently fails on main when the packaged reference runner aborts on the existing x + 1 model, and the test job is skipped while that is true. Both happen before this new code runs.
ExecuTorch 1.4.1 ships no linkable C++ runtime: its wheel contains zero shared libraries, its CMake package exports only a static _portable_lib, and no CUDA wheel exists for it on any channel. That is why the runtime wheel rebuilds ExecuTorch from source today, and it is the blocker for shipping only the TensorRT delegate. The prebuilt runtime landed on ExecuTorch main on 2026-08-20, six days after 1.4.1 was tagged, so no release carries it yet. Move the pin to the nightly line that does, keeping the release-line range on installable metadata so the same range prefers 1.5.0 over any dev build the day it ships, with no edit needed. The two pins now have to name one ExecuTorch rather than two that look close, because the delegate compiles headers from the source tree and links the runtime out of the wheel. Every wheel records its source commit, so add a test asserting the pinned commit is the pinned wheel's own git_version. Nothing else was enforcing that, and a mismatch is silent: both pins look plausible and the build succeeds. Deriving the range with a three-field split raised on the nightly form, so derive it from the release line the first two fields name. ExecuTorch's CUDA wheels are published only on the PyTorch nightly index, so every install site gains that channel. No site gains --pre: the pin names an exact dev build, which pip installs from an explicit version without it, and the existing --pre uses here are for torch. CI derives the channel from the row's own CU_VERSION, which keeps the runtime the delegate links to the same CUDA build as the rest of the job.
The CI installer globs torch_tensorrt*.whl, which also matches the ExecuTorch runtime wheel, whose install_requires names a dev build published only on the nightly channel. With no index on that line the whole pip invocation failed, and because line 1's set -e is commented out the failure was swallowed and the job died later with a confusing ImportError. The two range sites installed a range against the nightly channel, which gains a member every day, so they resolved to whatever was newest while the delegate is compiled from the commit the pin names. Both now request the pin exactly, which is the pairing the drift test exists to check; it was written to skip in exactly the state the ranges produced, so nothing reported it. setup.py keeps its range, because a published requirement has to stay resolvable for users off the same line. The two shapes now differ deliberately and test_derived_requirements_match_the_pin checks each for its own. Six printed install instructions gave a bare pip install of the executorch extra, which cannot resolve a dev pin from PyPI. They name the channel now. The discovery regex saw only == and >=, so a site added with any other PEP 440 operator was invisible to the drift check. It now recognises all of them.
The executorch suite is nightly-only, and the nightly matrix runs cu132 rows as
well as cu130 ones, so the fixed cu130 channel in tests/ci/runner.py would
install a CUDA 13.0 ExecuTorch into a CUDA 13.2 job. PR builds are pinned to
cu130 by filter-matrix.py, which is why watching PR CI could never show this.
Derived from CU_VERSION now, with cu130 as the local default, matching what the
two workflow files already did.
Three example docstrings still printed a bare `pip install -e ".[executorch]"`.
That resolved off PyPI before this pin moved to a dev build; it cannot now, so
they name the nightly index too. The runtime's ImportError advice and the
reference runner README already did.
The installer's hardcoded nightly channel gets the reason written down: a .dev
wheel exists on no other channel, so deriving it from ${CHANNEL} like the lines
above would break the install on exactly the test and release runs the index was
added for.…alls from Raising the executorch floor to a dev build made `uv lock` fail outright. The lock's win32 required-environment in pyproject.toml resolved the extra from PyPI, whose executorch stops at 1.4.1 -- uv.lock records five win_amd64 wheels for it -- so nothing satisfied the new range and uv errors rather than falling back. Reproduced against a probe project: without a marker uv reports the win32 split unsatisfiable, with one it resolves. The requirement now carries `platform_system == 'Linux'`, the shape EXECUTORCH_RUNTIME_REQUIREMENT already uses, which also stops pip reporting no matching distribution for Windows users of the extra. The delegate is a Linux object and ExecuTorch publishes CUDA wheels for no other platform, so the marker states what was already true. docgen installed the extra with --pre against the nightly channel, so it resolved through the range and took whichever dev build was newest that morning while the delegate compiled from the pinned commit. It names the pin now, read out of dev_dep_versions.yml. The pip line that installs both wheels gets `|| exit 1`. linux-test.yml concatenates this installer ahead of the user script and line 1's `set -e` is commented out, so a failure there was discarded and the job died later with an unrelated-looking ImportError; measured with `false` in place of the pip call, exit was 0 and the user script still ran. Two tests were checking source text rather than behaviour. The CUDA-row test now calls _setup_commands with CU_VERSION set and unset and reads the URL, which catches keeping the os.environ.get line while hardcoding the channel -- the mutation the string match passed. The drift check now asserts the set of files that pin ExecuTorch, because a site changing to bare `executorch` stops matching the search entirely and left the old `assert found` satisfied. Also corrects two claims: 1.4.1 does ship _portable_lib.so, so the comment says its executorch/lib carries no standalone linkable runtime, and no install site gains --pre, since an exact .dev pin needs none.
…tup fails The reference-runner README and the runtime's ImportError advice both printed `pip install "torch-tensorrt[executorch]"` with no index, and the commit that introduced the pin claimed otherwise. That claim was checked against the wrong branch: the fix existed only on the stacked runtime-wheel change, so this branch kept shipping the bare command. It matters more here than a docs nit, because this branch is what raises the floor above PyPI's newest executorch, so the bare command now cannot resolve at all. All seven printed install instructions carry the channel. The drift checks were counting the wrong thing. The requirement test asserted a set of paths, but two files carry two sites each, so either could drop one and stay in the set: turning `executorch-build-linux.yml:88` or `:128` into bare `executorch` both survived. The commit test only asserted nonzero, so any single MODULE.bazel could switch to `branch = "nightly"` unnoticed. Both now assert a per-file site count through one helper, as a minimum rather than an exact number so it holds on the stacked branch too, which removes one README site. All five mutations are caught and each names the file. Counting also surfaced a fifth commit site the nonzero check could not see: the reference-runner README's EXECUTORCH_REF shell default, correctly pinned but unaccounted for. docgen's pin was invisible to both: it is built by a shell substitution, so `$(` is not a digit and the literal search never saw it, and deleting the line survived. The derived-requirement test now runs the command docgen embeds and compares what it prints. A failed setup step printed `::warning::` and fell through to pytest. Most of the executorch suite gates on pytest.importorskip, so a failed ExecuTorch install skipped those files, left the rest passing, and reported success with a populated junit xml -- green exactly when the suite could not test what it exists to test. Driving the real run_suite with a failing setup step reproduced it, and returning the code makes it red without invoking pytest. Pre-existing, but this branch makes it likely to fire, since a nightly pin is eventually pruned from the channel. The pin tests themselves ran on nightly only, so none of this drift machinery ran on a PR or a push to main -- when a pin actually goes stale. They need no GPU, no ExecuTorch and not even torch, so they move to their own l0 suite in every lane, and the nightly suite excludes them by keyword so nothing runs twice. Also: the reference-runner README no longer says the extra installs the runtime wheel, since that requirement is commented out in setup.py.
…U lane The drift checks read derived strings and never the values CI consumes, so three ways of silently shipping no ExecuTorch all stayed green. Dropping the requirement from the runner's setup command left the step succeeding with nothing installed, after which the suite skips on importorskip; emptying EXTRAS_REQUIRE["executorch"] broke every documented `pip install "torch-tensorrt[executorch]"`; and the runtime README was recorded as carrying one pin site when it carries two, so either could go bare while the other satisfied the count -- the exact hole the per-file counts were added to close. The checks now assert the argument list the runner builds, the extras entries by AST, and the true per-file counts. All five mutations fail now. run_suite had no test at all, so replacing its `return rc` with `continue` restored the silent-green behaviour the fail-closed change exists to prevent. It is driven directly now, asserting both the propagated exit code and that pytest never runs once setup has failed. The pin suite was landing on a GPU runner: Suite.runner defaults to the matrix validation runner, so a five-second text check became one CUDA-container job per python and CUDA row, behind a wheel build. It runs in the Python lint job instead, which is already ubuntu-latest and needs none of that. The claim that it needs "not even torch" was also wrong -- tests/py/dynamo/conftest.py imports torch at module scope, which is why the lint invocation passes --noconftest. The shell tier that runs the whole executorch directory now excludes the pin file too, so the dedup claim is true of both paths rather than just the manifest one. uv.lock still records the pre-bump range with no platform marker. uv-update.yml regenerates it on pushes to main touching setup.py, and only that workflow runs `uv sync --locked`, so this breaks nothing -- but the drift was invisible, since the lock writes a bare specifier the pin search cannot match. A strict=False xfail records it and turns into a real failure via XPASS once the lock is refreshed. Editing the lock by hand was the wrong fix: its resolved entry and hashes come from a resolver run against the nightly index. Also removes internal shorthand from the PR description, and corrects a line citation for the one deliberate range in executorch-build-linux.yml.
The lint step added for these checks could not execute. It invokes pytest, and the job installs .github/scripts/requirements.txt (PyGithub) plus the lint dependency group (black, clang-format); neither carries pytest, so the step exited 1 on "No module named pytest" before running a single assertion. pyyaml is needed too, because reading the pin file shells out to a yaml import. Both are installed now, and a test asserts the step exists and installs them, since deleting it is otherwise invisible: every assertion here still passes locally while nothing runs it on a pull request. Reproduced the failure in a stdlib-only venv and confirmed the fixed command passes with only those two. The step also gets if: always(), so an unrelated formatting failure earlier in the job no longer hides the pin check. Three properties the checks are supposed to protect had no coverage: Deleting both published extras from EXTRAS_REQUIRE left everything green. The loop iterated whatever keys existed, so removing them iterated nothing and was indistinguishable from them being correct. It now requires the two published keys to be present, and only those, which also stops an unrelated future extra from turning this red for naming no ExecuTorch. The workflow opt-out marker was ordinary prose, "verify the end user's workflow". Pasting that sentence above a requirement and widening it to a range passed. It is an explicit token now, and the upward scan walks through comment lines to find it, so a cosmetic line between the opt-out and the requirement neither reclassifies the site nor fails the build. Nothing asserted that printed install instructions name the nightly channel, which is why that regressed and was re-fixed three times in this change without anything noticing. One test covers all of them by reading whole blocks rather than single lines, since every instruction wraps and the index lands on a continuation. It catches the CI install of the locally built wheel too, which carries no extra and is the site that broke most often. Generated docs under docs/ are excluded: corrections belong in docsrc/, and the committed Sphinx output is stale there independently. Also: the executorch requirement now strips its local version label like the other four, so the wheel does not bind itself to one CUDA train; the lockfile xfail is strict, since a non-strict xfail reports XPASS and ignores it and so could never fail; the fail-closed comment says it covers every setup step rather than implying only executorch; an empty frozenset and the dead branch reading it are gone; and the sys.path mutations use monkeypatch so they do not leak between tests.
The check that the two pins name one ExecuTorch could not run anywhere. It skips unless the installed wheel is exactly the pinned version, so it means something only on the nightly GPU lane, and that lane deselected it. The deselection is written as "not test_executorch_pin" to skip the source-consistency checks in the same file, but -k matches the module name in the test id, so it dropped every test in the module including this one. Both deselection sites now keep it by name. Proved it on a host with the pinned wheel installed, whose recorded git_version is the pinned commit: the check passes at the correct pins, fails when the commit pin names a different tree, and fails when the commit pin is deleted outright. Before this it was deselected in all three states. Bumping the version alone still skips, correctly, because the installed wheel is then not the one the pin names and its provenance says nothing about whether the two pins agree. A test asserts both sites keep it, since re-tightening either one to a bare module name is a small and plausible edit that would silently restore the gap.
Every guard added in this change asserted that a string appeared somewhere in a file, so each certified the state it was written to prevent. The keyword guard grepped for the kept test's name. Changing "or" to "and" in both -k expressions left it green, and that expression collects nothing at all, which is worse than the bug the guard exists to catch. Reverting the expressions and leaving the name behind in a comment also left it green, and a comment explaining the keyword sits directly above it, which is where an editor would naturally write that name. It now runs pytest's own collection under each expression and requires exactly the pairing test to come back. The CI guard searched the workflow as one blob, so it could not tell which job it was reading. The same commit that fixed the lint failure also added pytest and pyyaml to cpp-linting, which has no pin check, so deleting them from the job that does run it stayed green and would have restored the original failure invisibly. Neutralising the command while leaving its filename in a shell comment, and setting a falsy step condition, were also green. It now parses the workflow, finds the job that actually invokes pytest on this file, and requires the installs in an earlier step of that same job. The unused installs are gone from cpp-linting. The requirement pattern captured an equality prefix and stopped, so "executorch==PIN,!=PIN", a specifier that excludes the version it appears to pin, compared equal to the pin. The same truncation rejected the legal PEP 508 spelling with spaces around the operator. Requirements are parsed now and compared as specifier sets, with a check that the pinned version actually satisfies them. The site scanner counted raw search hits, so gutting a pin to a bare "executorch" while putting the exact pin in a comment in the same file kept the per-file minimum satisfied. Comments no longer count, except in the bazel repositories, where the annotation beside the pinned commit is the only record of which wheel that commit belongs to. Also corrected two claims this change made: the executorch tier is reachable from a pull request through executorch-test-linux.yml as well as the nightly manifest, so it is not the only route, and the shell helper now says why one test is kept out of the deselection.
The uv.lock check was a strict xfail. uv.lock records ">=1.4.1,<1.5" while the pin derives ">=1.5.0.dev20260822,<1.6", so the assertion fails and the xfail is satisfied. Refresh the lock and the assertion passes, and a strict xfail reports that pass as a failure. The lint step runs this file with if: always() on every pull request, so one lock refresh would have made the lint job red on every subsequent pull request, for a file none of them touched, until someone edited this test. Measured: baseline 1 xfailed, and 1 failed once the specifier is bumped. My own docstring claimed the lock is machine-generated and not edited by hand. Two hand refreshes landed on 2026-08-23, inside ordinary version-bump changes, so that was wrong as well. It now accepts both resting states and only fails where something is actually wrong: a recorded range whose lower bound is above the pin, which means the lock names an ExecuTorch this repository does not pin. Behind the pin passes, the derived range passes, and ">=1.6,<1.7", an open-ended ">=1.7" and "==1.9.0" all fail. Comparing lower bounds rather than probing the specifier with sample versions: an upper-bound test missed the open-ended case, and a low sentinel version called the ordinary behind-the-pin state a failure.
test_derived_requirements_match_the_pin extracted the python3 -c one-liner from docgen.yml and ran it. Whatever that line said got executed on every pull request: rewriting it to write a file left the test green and the file written. Same class as the bash -c problem fixed in test_api.py last round, still live here. It now compares the command as text against the exact form that reads __executorch_version__ out of dev_dep_versions.yml. Four mutations caught, including a payload that writes a file and still prints the right version, with nothing executed. The CI reachability guard tested the raw string for "--collect-only", so it accepted "--co", pytest's own documented short form, which collects and asserts nothing. It also could not see an exit status being discarded. Now tokenised: --collect-only, --co, -h, --help, a "||" short-circuit and continue-on-error are all rejected, and all five are caught where four previously survived. The comment exemption for .md/.rst/.txt defeated exactly the threat its docstring names. Install commands live in prose files, so exempting them made a comment count as a pin there: the runtime README's install line gutted to a bare "executorch" passed as long as a decoy "# executorch==<pin>" sat beside it, and failed only with no comment present. The exemption is gone, and trailing comments no longer count either, since a decoy after a live requirement on the same line kept the per-file count satisfied. Five mutations caught, baseline green.
…it resolves The nightly-index guard matched only the named-distribution spelling, so the four sites that write "pip install .[executorch]" were unguarded: docgen.yml and the three export examples. The nightly index could be deleted from all four with the test green. Each of the four is now caught individually. Its second half was a bare substring test for the host, which proves a string sits nearby rather than that the instruction resolves. Rewriting every channel in the tree, 18 files, to a nonexistent cu999 left it green. The CUDA suffix is now checked against the set the project publishes for. Deliberately not compared against __cuda_version__: five sites legitimately say cu130 while the pin says 13.2, and I confirmed against the live index that cu130 and cu132 both carry 38 ExecuTorch wheels while cu999 carries none.
The printed install commands resolved no ExecuTorch. "torch-tensorrt[executorch]" with no version pin resolves the stable PyPI wheel, which carries no executorch extra, so the command exited 0 and installed nothing the feature needs. Add --pre to the six commands that name the extra and assert its presence in the guard that already reads them. Close four ways to neutralise the pin check while its guard stayed green: a ";" or "&" terminator after pytest, continue-on-error or a falsy if: on the owning job, and reducing the workflow trigger so it never runs on pull requests. The trigger check also handles PyYAML reading the unquoted "on" key as the boolean True. Close both ways to strip the pairing check while its guard stayed green: assert the workflow actually calls trt_tier_executorch, and validate suite lane names against the known set so a typo raises at import instead of silently dropping the suite from every matrix. Also: anchor the docgen pin check to a live line so a commented-out install no longer satisfies it; fix the lockfile range check crashing on a legal "==1.4.*" clause; correct the range comment to describe what the range admits; and note in the install advice that the feature is published for Linux only.
shoumikhinforce-pushed
the
executorch-pin-dev-line
branch
2 times, most recently
from
August 30, 2026 00:48
4ad4825 to
ba73bbaCompareThe delegate is built against one ExecuTorch: __executorch_version__ selects the wheel it links against and __executorch_commit__ selects the tree it compiles from. Those two values repeat across the build workflows, the bazel modules, the docker and toolchain copies, and the docs, so they can drift apart or fall behind upstream with nothing to notice. Add a script and a daily workflow that move both pins to the newest ExecuTorch wheel on the nightly index. The source commit is read from the chosen wheel's own version.py, so the two pins always name one ExecuTorch rather than two that happen to be close. The update lands as a pull request, so the pin consistency checks and the delegate build and test lane decide whether the new wheel is usable before it reaches main. A day with no new nightly rewrites nothing and opens nothing. On a release branch the schedule is a no-op and the pin moves only by a manual run pointed at the stable line, so a cut release does not drift. Back the mechanism with consistency checks that run under the linter. Every requirement and comment that names ExecuTorch is asserted to match the pinned version, including the variable-index install once the variable's assignment is resolved and extensionless install files like justfile. The source commit is checked against the wheel's own provenance wherever that wheel is installed, and commits left in comments are not mistaken for pins. The wheel-content and CI-invocation checks measure effect, running the workflow's own step against a passing and a failing stub and requiring the exit status to follow, rather than enumerating bypass spellings. Install the built wheel in the runtime README rather than an unpublished package. The guard that checks the pin runs in CI compares the step's command as text rather than executing it. Running the step's own shell body meant whatever that body said ran on every pull request: appending a line that writes a file left the test green and the file written. That is the same defect this file already avoids for the docgen one-liner, and the reasoning there applies here too.
shoumikhinforce-pushed
the
executorch-pin-dev-line
branch
from
August 30, 2026 04:19
ba73bba to
d765679Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Torch-TensorRT can run part of a model with TensorRT inside ExecuTorch. The piece that does this is called a delegate. To build the delegate, the project has to point at one exact version of ExecuTorch. That pointer is called the pin.
This change does two things:
executorch==1.4.1onto a recent ExecuTorch nightly build.The problem
The old version cannot be linked against
The delegate is a shared library. It needs to link against
libexecutorch.so, the C++ code that runs a model.ExecuTorch 1.4.1 does not ship that library in a usable form. Here is what you get in each wheel:
So with 1.4.1 you had to rebuild all of ExecuTorch from source before you could build the delegate. With a recent nightly you install the wheel and link against it.
A hand-written pin goes stale
ExecuTorch publishes a new build every night. A pin someone bumps by hand falls behind, and then the delegate is only ever tested against old code. Somebody has to remember to move it, and that gets forgotten.
The fix
The pin is two values that must agree
The delegate needs two things:
__executorch_version__: which wheel to install and link against.__executorch_commit__: which source tree the headers come from.Both live in
dev_dep_versions.yml:Why they must move together: the delegate compiles against headers from the commit, then loads the runtime from the wheel. If those are different versions, the two sides can disagree about how a struct is laid out in memory. That is called an ABI mismatch. It does not give you a nice error. You get something like this at load time, or a crash:
Every published wheel records the commit it was built from, in
executorch/version.py:So the pairing is checked against the real wheel instead of trusted.
The daily job
A scheduled workflow runs a few hours after the nightly index publishes:
If the newest nightly is already pinned, it does nothing and opens nothing. It opens a pull request rather than pushing, so the move has to pass CI before it lands.
The newest wheel on the index is the newest one that built. If a night failed, that build is simply absent, the newest version does not change, and the run does nothing. There is no special case for a failed night.
Guards
Two test files keep this honest.
The first checks every place that names an ExecuTorch version. There are ten, and they drift apart easily:
It also checks the pinned commit really is that wheel's own source, when the pinned wheel is installed. The updater refuses to run if a pin site is not tracked by git, so a moved file fails loudly instead of being skipped.
The second tests the update script itself: that it rewrites every pin site, leaves other packages alone (a line like
bazel_skylibversion1.7.1must not be touched), and moves every requirement shape the guard counts.Known gap
The published extra. Before this change,
torch-tensorrt[executorch]resolvedexecutorch>=1.4.1,<1.5with no platform marker, and installed on Linux, Windows and macOS from PyPI. This change makes it>=1.5.0.dev20260829,<1.6with a Linux marker, because the runtime split the delegate needs does not exist in any ExecuTorch release yet.PyPI carries no ExecuTorch 1.5.x, so on Linux
pip install torch-tensorrt[executorch]now finds no candidate and fails. On Windows and macOS the marker makes the extra resolve to nothing, so the install reports success and the failure moves to import time.Worth deciding whether the published extra should track the release line while only the development pin follows the nightly index.
Test plan