Skip to content

Add the MLX backend product to the SwiftPM package template - #22261

Merged
shoumikhin merged 7 commits into
swiftpmfrom
swiftpm-mlx-metallib-fix-v4
Aug 28, 2026
Merged

Add the MLX backend product to the SwiftPM package template#22261
shoumikhin merged 7 commits into
swiftpmfrom
swiftpm-mlx-metallib-fix-v4

Conversation

@shoumikhin

Copy link
Copy Markdown
Contributor

Supersedes #22204. Same change, plus the fix for a defect that made the new resource bundle empty for every real consumer.

What #22204 got wrong

The metallib resource list was gated on FileManager.default.fileExists with a path relative to the process working directory:

letmlxResourcesDir=".Package.swift/backend_mlx_resources"resources: mlxMetallibSlices.compactMap { slice inFileManager.default.fileExists(atPath:"\(mlxResourcesDir)/\(slice).metallib")?.copy("\(slice).metallib"):nil}

SwiftPM evaluates a dependency's manifest with the consumer's directory as the working directory, not the dependency's checkout. So every slice looked absent, compactMap returned [], and SwiftPM emitted no bundle at all. The build stayed green with no warning, and the MLX delegate would fault later at Metal device init inside a consumer's app with nothing pointing at the cause.

Measured from a real consumer package, with all three metallibs committed and present in the resolved checkout:

before: Build complete! bundles: 0 metallibs shipped: 0
after: Build complete! bundles: 1 metallibs shipped: 3
.build/out/Products/Debug/executorch_backend_mlx_resources.bundle

That bundle name is what backends/mlx/CMakeLists.txt compiles in as SWIFTPM_BUNDLE, so it is the name the delegate actually looks for.

Why it was easy to miss

Every in-place check passes. swift package dump-package run inside the package root reports all three resources, and --package-path chdirs to the package root too, so it is equally blind. Only resolving the package as a dependency exposes it. The release job's own guards check for unsubstituted checksums and for products without frameworks, neither of which can see this.

The fix

Declare the three slices unconditionally instead of probing for them. The release job commits all three files to this branch before publishing, so the probe was guarding against a state that does not occur on a published branch. import Foundation is no longer needed, and the base template did not have it.

What a genuinely missing slice does now, measured rather than assumed:

swift package resolve -> says nothing at all
swift build -> warning: 'dep': Invalid Resource 'mlx-ios.metallib': File not found
Build complete! (a warning, not an error)

So this makes the failure visible, not fatal. That is still a large improvement over the old form, which shipped zero of three metallibs with no diagnostic of any kind, but it is worth being precise: SwiftPM does not fail a build on a missing declared resource.

Landing

This half and the main-side half (#22203) both need to be live before the next Apple publish run. Order between them does not matter: each single-sided state trips one of the release job's two guards and exits 1 before git push -f, so nothing broken is published either way. Verified by replaying the substitution and both guards for all three states.

Changes since #22259 and #22260

Two review findings on the comment itself, both measured before the edit:

  • The comment said not to gate on FileManager.fileExists. The main-side manifest does exactly
    that and is correct, because it anchors the path with #filePath. As written, the two halves of one
    feature gave opposite instructions about the same API. The comment now names the real pitfall, a bare
    relative path, and points at the anchoring.
  • The single-use mlxResourcesDir local is inlined, matching how every other target in this manifest
    writes its path.

Re-verified after both edits: dump-package and describe both rc=0, the resource target still
declares all three slices, both MLX products still share the one bundle target, and a real consumer
build still ships 3 of 3 metallibs into executorch_backend_mlx_resources.bundle.

Changes since #22260

The comment claimed a missing slice surfaces as a warning at build time. Measured under both drivers,
that is only half true:

swift build -> warning: 'dep': Invalid Resource 'mlx-ios.metallib': File not found
xcodebuild -> printed under "Resolve Package Graph", no warning: prefix at all
both -> BUILD SUCCEEDED

Under Xcode, which is what real apps use, it is not classified as a warning, so it never reaches the
issue navigator or the warning count. The comment now says only what holds for both drivers: the
absence is reported at package-resolution time and does not fail the build, so the release job has to
assert the files arrived.

The paragraph advising how to anchor a probe with #filePath is also gone. This manifest has no probe;
that advice belongs beside the main-side manifest, where the probe actually lives and where the
comment explaining it already is.

The published SwiftPM manifest is generated from this template, so the platform
floor a consumer sees is set here, not in the main branch's Package.swift.
Raising it to macOS 14 is what lets the MLX backend be offered through the
package. MLX requires a macOS 14 deployment target and fails to build below it,
so the package that ships MLX must declare at least that. iOS is already at 17,
which is above MLX's iOS floor, so only macOS moves.
This drops support for macOS 12 and 13 for the whole package, including consumers
that only use XNNPACK or Core ML. macOS 14 shipped in September 2023.
Test Plan:
Materialized the template the way the release job does, substituting the version
and checksum placeholders, and confirmed `swift package dump-package` parses it and
reports the macOS platform as 14.0.
Companion to the change that adds MLX to the Apple frameworks. The published
SwiftPM manifest is generated from this template, so the backend_mlx product and
its resource bundle have to be declared here too, or the released package would be
missing them.
Adds the backend_mlx product with its Metal, Foundation and QuartzCore framework
links, and a shared backend_mlx_resources target that carries the per-platform
Metal kernel libraries. The release job commits those libraries to this branch
beside the manifest. Each is included only when present so the manifest still
resolves before the release job has produced them.
The resource list was gated on FileManager.fileExists with a path relative to
the process working directory. SwiftPM evaluates a dependency's manifest with
the consumer's directory as the cwd, so every slice looked absent, the list came
out empty, and no bundle was produced at all. The build stayed green and the MLX
delegate faulted later at Metal device init, with nothing pointing at the cause.
Measured from a consumer package: 0 of 3 metallibs shipped before, 3 of 3 after,
in executorch_backend_mlx_resources.bundle, which is the name the delegate is
compiled to look for. An in-place build and swift package dump-package both
reported the resources correctly either way, which is why this was not caught.
The release job commits all three files before publishing, so declare them
unconditionally rather than probing for them. Foundation is no longer needed.
The comment said a missing slice is reported at resolve time. Measured: swift
package resolve emits nothing at all, and the "Invalid Resource ... File not
found" warning appears when a consumer builds. It is also only a warning, so the
build still succeeds; the change makes the failure visible, not fatal.
Every other target in this manifest inlines its path, and this local was
referenced once.
The comment said not to gate on FileManager.fileExists, but the main-side
manifest does exactly that and is correct, because it anchors the path to
#filePath. Name the real pitfall, a bare relative path, and point at the
anchoring instead of forbidding the API.
The comment said a missing slice shows up as a warning when a consumer builds.
Measured under both drivers: `swift build` prints it with a `warning:` prefix,
but `xcodebuild` prints it during Resolve Package Graph with no prefix at all,
so it is not classified as a warning and does not reach the issue navigator or
the warning count. Both drivers still report BUILD SUCCEEDED.
Say what holds for both: it is reported at package-resolution time and does not
fail the build. Also drop the paragraph advising how to anchor a probe that this
manifest does not have; that advice belongs beside the main-side manifest, which
is where the probe actually lives.
@pytorch-bot

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22261

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 28, 2026
@shoumikhin
shoumikhin merged commit 5f8a349 into swiftpmAug 28, 2026
12 of 17 checks passed
@shoumikhin
shoumikhin deleted the swiftpm-mlx-metallib-fix-v4 branch August 28, 2026 22:41
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shoumikhin@Gasoonjia