Uh oh!
There was an error while loading. Please reload this page.
Add the MLX backend to the Apple frameworks and SwiftPM package - #22203
Merged
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22203
Note: Links to docs will display an error until the docs builds have been completed. ❌ 14 Unclassified FailuresAs of commit 085a1d6 with merge base 0447246 ( UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a |
Gasoonjia
approved these changes
Aug 27, 2026
- Build MLX for the correct per-slice deployment target. The iOS toolchain leaves CMAKE_OSX_DEPLOYMENT_TARGET at 12.0 on the iOS slices while the preset's DEPLOYMENT_TARGET holds the real minimum (iOS 17, macOS 14). The shader-flag patch reads the former, so it stamped the metallib with the wrong minimum. Feed the preset value into the sub-build as CMAKE_OSX_DEPLOYMENT_TARGET. - Fail the framework build if a metallib slice is missing. The copy loop skipped a missing slice silently, which would ship a package that resolves and then throws at first device init on that platform only. Assert each enabled slice produced its metallib, matching the framework-set guard. - Update the macOS floor in the docs starter snippet to 14, matching the package bump, so a consumer copying it still resolves. - Clarify the SWIFTPM_BUNDLE / MLX_SWIFTPM_METALLIB_NAME comment: they are a bundle name and a per-slice file name that MLX concatenates, not a fallback chain. Test Plan: Configured the macOS preset with the Xcode generator and confirmed configure and generate complete with these changes.
The resource guard for the MLX Metal kernel bundle checked the files with a path relative to the process working directory. When the package is used as a dependency, the manifest runs with the working directory set to the consumer's root, so the check was false for every slice and the bundle shipped empty, which made MLX fail to find its kernels at first device init. Anchor the check to this manifest's own directory with #filePath, so it holds whether the package is the root or a dependency. Test Plan: Ran swift package dump-package from a different working directory with the metallibs present and confirmed all three slice resources are included, where the relative check returned none.
The Apple framework build merges each static library straight from the target output directory without an install step. The MLX runtime archive is produced by the MLX sub-build in its own binary directory, not beside the delegate, so the packaging failed with "File ios/Release/libmlx.a does not exist" while merging the backend_mlx framework. Copy the archive next to the delegate archive after the delegate builds, into the per-config target output directory the packaging reads. The wheel build is unaffected because it installs the archive explicitly. Test Plan: Built the delegate with the Xcode generator and an archive output directory set the way the framework build sets it, and confirmed libmlx.a lands beside libmlxdelegate.a in the per-config directory the packaging merges from.
The resource filter this change added to the per-product target initializer pushed that single chained expression past the Swift manifest compiler's type-check budget, so the Apple framework build failed to compile Package.swift. Bind the dependency, resource, and linker lists to typed locals first, then build the target from them. Test Plan: swift package dump-package now completes in a few seconds and resolves all 18 products, where the compiler previously reported it could not type-check the expression in reasonable time.
The Swift test bundles force-load C++ static archives, so they need libc++. They never declared it, and below macOS 13 a Swift back-deployment shim pulled it in by accident. Raising the package floor to macOS 14 drops that shim, so the test link failed with many undefined C++ standard library symbols. Link libc++ explicitly in the shared test linker settings, which covers every test target. A product's libraries do not reach a test target, since a test bundle is its own linked image. Test Plan: Reproduced with a test target that force-loads a C++ archive: it links at macOS 12 and fails at macOS 13 and above, and linking libc++ makes it pass again.
…s, drop redundant Foundation Sort backend_mlx between backend_coreml and backend_xnnpack in the apple.yml FRAMEWORKS list and in both the definition and append order in build_apple_frameworks.sh, matching the rest of the backend group. Drop the Foundation linked framework from the backend_mlx product. It links by default on Apple platforms and no other product lists it. Trim the added comments to match the surrounding style: the sibling framework entries carry none, and the longer blocks repeated what the code already shows.
…floor The Apple framework build compiles each preset twice, once in Release and once in Debug, into the same directory. The step that copies the MLX Metal kernel file into the SwiftPM resource bundle ran after both, so it always shipped the Debug kernels. Copy the file during the Release pass instead, before the Debug pass overwrites it, so the published bundle carries the Release kernels. Also update the iOS docs: add the MLX backend to the list of shipped frameworks, which had every backend but this one, and state the iOS 17 and macOS 14 minimum so a consumer knows the required platform versions before resolving the package. Test Plan: Ran the Apple framework build and confirmed the resource bundle receives the Release metallib for each slice, and that a build with MLX turned off still skips it. Verified the capture picks Release across mode orderings and the Debug-only fallback with a shell trace.
The comment trims in the previous review pass left two lines over the 80 column limit, which lintrunner flags as a CMAKEFORMAT warning.
Removing it in the previous review pass was wrong. The MLX delegate's ObjC++ Metal code calls into Foundation, and a static archive inside an xcframework carries no autolink hints for the consumer's link line, so the framework has to be named explicitly. Linking Metal alone leaves _NSClassFromString and ___CFConstantStringClassReference undefined. This also brings the manifest back in line with the delegate's own CMake, which finds and links Foundation, and with the SwiftPM template.
The MLX external build was always passed -DCMAKE_CXX_FLAGS=, even when the value was empty, which is every non-Apple build including the wheel. An empty -DCMAKE_CXX_FLAGS= on the command line overrides the environment, so a wheel build's CXXFLAGS were silently dropped for MLX only. Pass the argument just for the Apple presets, where it carries the resource-bundle defines; leave it off otherwise so the environment is honored. Test Plan: Configured a child project with CXXFLAGS set in the environment and confirmed the flags survive when the argument is omitted and are cleared when an empty one is passed. Confirmed the argument expands to one entry on Apple and drops out cleanly elsewhere.
Three small corrections found in review. The iOS docs said a too-low platform version makes dependency resolution fail; it is the build that fails, with a message that the target's platform version is too low. Reword to match. The comment on the libmlx.a copy claimed it is correct under every generator; it relies on the multi-config Xcode generator the Apple presets use, so say that. The captured metallib lives outside the build output directory, so the top-level clean does not remove it. Remove any previous copy before writing the new one so a stale metallib cannot survive into a later build and ship. Test Plan: Ran cmake-format and bash -n on the changed files. Confirmed the metallib capture overwrites cleanly and the docs render.
Two problems with the MLX external build, both from forcing it to a single-config generator. A single-config generator reads CMAKE_BUILD_TYPE, not the --config passed to the parent build. The Apple presets do not set CMAKE_BUILD_TYPE, so a plain `cmake --preset ios && cmake --build cmake-out --config Release` compiled MLX with no optimization flags at all. Default the sub-build to Release when the parent left the type empty; the framework script and the wheel set it explicitly and are unaffected. The patch step is stamped, and BUILD_ALWAYS re-runs only the build step, so a reused build directory whose MLX source was reset would recompile an unpatched MLX and silently drop the iOS Metal SDK selection and the SwiftPM metallib name. Re-apply the patches on every configure through an always-run step; apply.sh is idempotent and skips patches already applied. Test Plan: Confirmed the build-type default resolves to Release when CMAKE_BUILD_TYPE is empty, and that the always-run patch step is accepted by a real cmake configure.
The MLX sub-build was defaulted to Release whenever the parent left the build type empty. Under the multi-config Xcode generator the Apple presets use, the type is empty for a Debug build too, so a bare-preset Debug build linked an optimized, assertion-disabled MLX. Follow the active configuration through a generator expression, which resolves to the config being built under a multi-config parent and to the build type under a single-config one, and fall back to Release only when it is genuinely empty (a bare single-config parent with no type set). Test Plan: Reproduced with a child ExternalProject under an Xcode parent: --config Debug now yields Debug and --config Release yields Release, while a bare Unix Makefiles parent with no type falls back to Release.
Document the MLX backend on the iOS page properly: mention MLX alongside Core ML in the opening line, scope the MLX bullet to macOS on Apple Silicon to match the C++ guide, add the MLX archive to the force-load example, and note that a source integration must also copy the MLX Metal kernel file, which is not among the frameworks in cmake-out. Remove the generic product loop's resource reader: no product declares a resources key, so it was dead, and it used the bare relative path that fails for a consumer, which would mislead anyone who later added one. The MLX metallib bundle is handled by its own target. Test Plan: Manifest parses. Reviewed the rendered docs.
Three issues with the MLX Apple build. The re-apply-patches step depended on the download step, but the built-in patch step also runs between download and configure, so a parallel build ran both against the same MLX checkout at once and they collided. Depend on the patch step instead, which orders the re-apply after it. The MLX sub-build ran fully serially under the multi-config Xcode parent, whose generated build step passes no parallelism and has no jobserver to inherit. Give it an explicit parallel build command. The Apple presets forced MLX on for every consumer, so an ordinary `cmake --preset ios` hard-failed on a machine without the Metal compiler or the MLX submodule, and plain PR CI built MLX three times. Probe for the Metal compiler and enable MLX only on Apple Silicon when it is present, degrading with a message otherwise, the way the wheel's pybind preset already does. Test Plan: Verified the metal-compiler probe enables MLX when present. cmake-format clean.
The comments claimed a Debug pass would overwrite a Release metallib and that the shipped kernels should be the Release build. The metallib does not depend on the build type, so the two passes emit the same file and there is no meaningful overwrite. Reword to say the capture runs once from a single pass because the file is build-type independent. No behavior change. Test Plan: Comment-only; confirmed the script still parses.
The previous change guarded the Metal-compiler probe with `CMAKE_SYSTEM_PROCESSOR STREQUAL "arm64"`, but under the Apple presets that variable is the target the iOS toolchain set, which is `aarch64` for Apple Silicon and never `arm64`. The guard was therefore always false, so MLX was silently disabled on every Apple preset, defeating the purpose of the build and announced only by a status message. Drop the processor condition and let the Metal-compiler probe be the whole gate, the way the wheel's pybind preset does. Test Plan: Confirmed the toolchain sets CMAKE_SYSTEM_PROCESSOR to aarch64, and that the probe alone enables MLX on a machine with the Metal compiler present.
The note named `mlx.metallib` and told the reader to copy it into the app bundle, but the shipped library looks for a per-slice `mlx-<slice>.metallib` inside a bundle named `executorch_backend_mlx_resources`, and the file it pointed at in the build tree is removed by the build's own cleanup. Point at the correctly named files the build stages under `.Package.swift/backend_mlx_resources/` and describe the bundle the runtime actually reads, so following the note prevents the no-kernels failure instead of causing it. Test Plan: Documentation only. Cross-checked the file names and bundle name against the build script and the MLX metallib-name compile flag.
The comment gave two wrong reasons for linking libc++ into the test bundles. The setting is necessary and correct; only the explanation was off. The tests link libc++ because they depend on the executorch binary target directly, which carries no linker settings, not the with-dependencies target that owns the link, and the implicit supplier that disappears at this package's floor is the Swift shim that existed below a macOS 13 deployment target, not 14. Comment only. Test Plan: Manifest parses.
Three separate defects found by review, each measured. The sub-build's BUILD_COMMAND passed a bare --parallel, which expands to `make -j` with no job limit. It also applied unconditionally, and under a single-config parent it replaced ExternalProject's default `$(MAKE)`, which already inherits the parent's jobserver, so it broke that and made MLX build serially on the wheel path. Measured: jobserver-unavailable warnings went 0 to 1 with the override. Now it names a job count and only overrides for a multi-config parent, where there is no jobserver to inherit. The reapply_patches step used DEPENDERS configure, which invalidates the sub-build's configure stamp on every build, so MLX re-ran its whole configure, including the metal_cpp FetchContent, on every incremental build of all six slice and mode combinations. Measured with the step: apply and configure both ran on builds 1, 2 and 3; with DEPENDERS build, configure runs once. Its comment also described depending on PATCH_COMMAND, which this branch no longer sets. capture_mlx_metallib decided "MLX is enabled" from CMAKE_OPTIONS_OVERRIDE, which is empty unless a --<backend> flag was passed. The Apple presets probe for the Metal compiler and leave MLX off when it is missing, so a plain no-flag build on a machine without the Metal toolchain hard-failed with a message asserting MLX was enabled. Read the option out of the preset's CMakeCache.txt instead. Also drop the "(macOS on Apple Silicon)" qualifier from the docs bullet: the build produces all three slices, a missing iOS metallib is a hard error, and no sibling entry carries a platform parenthetical.
The probe added in the previous commit grepped for EXECUTORCH_BUILD_MLX:BOOL=ON, but the preset path writes STRING: set_overridable_option uses `CACHE STRING ""`, and that is what apple_common.cmake calls. Measured against a real preset chain, the cache holds EXECUTORCH_BUILD_MLX:STRING=ON, so the grep never matched and the guard skipped the metallib capture even when MLX was enabled, which is the inverse of the intended behaviour. Match any cache type and accept CMake's truthy spellings. Verified for STRING=ON, BOOL=ON, STRING=1 and STRING=TRUE (proceed) against STRING=OFF and BOOL=OFF (skip).
…a too Two regressions from my own previous commit, both found by review. DEPENDERS build ran the MLX sub-configure on unpatched sources. Measured: with DEPENDERS build the order is SUB_CONFIGURE UNPATCHED then the patch step; with DEPENDERS configure the patch lands first. That matters because mlx_metal_sdk_per_platform.patch edits the sub-project's CMake to pick the Metal SDK from PLATFORM, so configuring before it is applied generates shaders against the macOS SDK for every slice, which is the exact bug that patch exists to stop. Reverted to DEPENDERS configure and said in the comment why the per-build reconfigure is the price of the ordering. The build-command override was gated on CMAKE_CONFIGURATION_TYPES, which sends a Ninja parent down the empty branch. ExternalProject emits $(MAKE), and so inherits the parent's jobserver, only when the PARENT generator matches "Make"; under Ninja it emits a bare `cmake --build` with no -j, so MLX compiled on one core. Gate on the generator instead. Also drop --config $<CONFIG> from the override, which is inert against the forced single-config sub-build.
…hing Four related gaps, each measured. append_framework_flag still decided from the command-line flags, so a no-flag build on a machine whose preset disabled MLX asked create_frameworks.sh for libraries that were never built, and it exits 1 on a missing input. The previous commit fixed only the capture side. Both now read the same cache value through one helper. capture_mlx_metallib created the resources directory before deciding whether it had anything to put there, so a disabled build left an empty directory behind. The publish step tests for that directory to decide whether to commit metallibs, so an empty one made the test pass and the copy silently no-op. Create it only once there is a file to copy. The publish step had no check that the metallibs arrived, while the copy that puts them there is conditional and the capture upstream is best effort. Assert all three are present and non-empty whenever backend_mlx is being published, beside the two checks already there for the same class of silent failure. Nothing evaluated the manifest before consumers did: the existing checks are text greps, the swiftpm branch has no CI, and a .template is not a manifest. Run dump-package on the substituted file so a Swift-level defect fails the run instead of every consumer of the published branch.
lintrunner flagged CMAKEFORMAT on this file. Align the continuation with cmake-format's argument-aligned style under dangle_parens.
Taken verbatim from the lintrunner diff rather than guessed: cmake-format reflows comment paragraphs, and both blocks had a short line mid-paragraph.
This was referenced Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds the MLX backend to the ExecuTorch SwiftPM package and Apple frameworks. MLX runs models on the Apple GPU through Metal. Before this, MLX was only reachable from the pip wheel; now a Swift or C++ app can use it the same way it already uses the Core ML and XNNPACK backends.
What this needed
1. Raise the package minimum to macOS 14. MLX requires a macOS 14 / iOS 17 deployment target and does not build below it, so a package that ships MLX must declare at least that. iOS was already at 17. This drops macOS 12 and 13 for the whole package (macOS 14 shipped September 2023).
2. Deliver the Metal kernel library per platform slice. MLX loads a
metallib(compiled Metal shaders) at runtime, and a metallib built for one slice does not load on another. So one is built for iOS device, iOS simulator, and macOS, and all three ship together in a single SwiftPM resource bundle (executorch_backend_mlx_resources.bundle). Each slice's MLX binary is compiled to ask for its own file, so the right one is always used.Two small patches to the vendored MLX build make this correct:
mlx_metal_sdk_per_platform.patch— MLX hardcodedxcrun -sdk macosxfor shader compilation; this selects the SDK from the target platform (iphoneos/iphonesimulator/macosx).mlx_swiftpm_metallib_name.patch— lets the runtime look up the per-slice file name instead of a single fixeddefault.metallib.3. Nothing else. Like the other backends, linking the framework registers MLX automatically, and it runs through the existing
ModuleAPI with no new Swift class.What changed
Package.swift+CMakePresets.json: macOS floor to 14;backend_mlxproduct; a sharedbackend_mlx_resourcesbundle target both the release and debug delegates depend on.tools/cmake/preset/apple_common.cmake: enable MLX for the Apple presets.scripts/build_apple_frameworks.sh: a--mlxflag; copy each slice's metallib into the resources target asmlx-<slice>.metallib..github/workflows/apple.yml: addbackend_mlxto the frameworks list; carry the metallibs in the build artifact and commit them to the package branch at release time.backends/mlx/CMakeLists.txt: register the two patches; compile in the resource bundle name and the per-slice metallib name for Apple framework builds.A companion change to the package template branch adds the
backend_mlxproduct and the resource bundle to the published manifest.Test plan
backend_mlxproduct and the sharedbackend_mlx_resourcestarget, that both delegates depend on the one bundle, and that the per-slice metallib resources are included when present and omitted cleanly when absent (so a fresh checkout still resolves).