Skip to content

Stop reading a backend config field that no longer exists - #22298

Merged
shoumikhin merged 4 commits into
pytorch:mainfrom
shoumikhin:llama-stale-mps
Sep 1, 2026
Merged

Stop reading a backend config field that no longer exists#22298
shoumikhin merged 4 commits into
pytorch:mainfrom
shoumikhin:llama-stale-mps

Conversation

@shoumikhin

@shoumikhinshoumikhin commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

The code fix this pull request opened with has since landed on main separately, so what is left here is
the test that keeps it from coming back.

The bug: exporting llama for Core ML failed before it started, because the branch selecting the Core ML
lowering read a backend config field that no longer existed.

AttributeError: 'BackendConfig' object has no attribute 'mps'

or short-circuits, so this raised only when Vulkan was off. Core ML alone and Core ML with QNN
raised; Core ML with Vulkan did not, because the first term was already true.

What is in this change now

Three tests, one per route through that branch, with the lowering patched so they need no macOS and no
coremltools:

caseroutes to
Core ML alonethe Core ML lowering
Core ML with QNNthe combined lowering, QNN partitioner kept
Core ML with Vulkanthe combined lowering, Vulkan partitioner kept

The second and third cases matter on their own. The exclusion clause exists to keep those two
partitioners, and the Core ML lowering takes neither as an argument, so routing there would drop them
in silence. Both cases assert the backend flags actually reach the shared lowering, since a call count
cannot see them.

The mutation matrix is complete:

drop the vulkan term the Vulkan case fails
drop the qnn term the QNN case fails
delete the whole clause both combined cases fail
restore the removed field both Core ML cases fail

The model preparation is stubbed, because routing is decided before the model is touched. That takes
the three cases from about 4.3 GB of peak memory to 420 MB. The KV cache is pinned off with a comment
saying why: with it on, the source transforms import the Qualcomm SDK, which routing does not need.

Test plan

All three pass against main as it stands now, and each fails on the mutation named above.

CopilotAI lite review requested due to automatic review settings August 29, 2026 05:13
@pytorch-bot

pytorch-botBot commented Aug 29, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22298

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 21 Pending, 1 Unrelated Failure

As of commit a589a92 with merge base beb5b16 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 29, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shoumikhinshoumikhin added the release notes: examples Changes to any of our example LLMs integrations, such as Llama3 and Llava label Aug 29, 2026
CopilotAI review requested due to automatic review settings August 29, 2026 11:17

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

CopilotAI review requested due to automatic review settings August 31, 2026 04:04

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shoumikhin

Copy link
Copy Markdown
ContributorAuthor

Good catch on the coverage gap. Fixed and pushed.

The tests pinned only the QNN half of the clause. I confirmed your mutation: dropping just the Vulkan
term left both green, and that sends a Core ML plus Vulkan export to the Core ML lowering, which
takes no Vulkan argument, so the partitioner is dropped in silence. A third case covers it. The
matrix is complete now:

drop vulkan term the Vulkan case fails
drop qnn term the QNN case fails
whole clause deleted both combined cases fail
stale field restored both Core ML cases fail

You were also right that the helper hid the Core ML flag. Deleting that line left the test named for
Core ML with QNN passing, so it never checked Core ML was on. Backends are named at each call site
now. That removed the getattr lookup as well, which was the same unchecked kind of lookup this
change deletes, and the f-string patch target is now patch.object.

The bare RuntimeError marker is gone too. Your kv cache experiment is the convincing part: a real
RuntimeError from inside the export was swallowed and the test reported only that the lowering was
never called. A private marker class shows the real cause.

Both combined cases now assert the backend flags reach the shared lowering, since a call count cannot
see them and preserving those partitioners is the whole point of the clause.

On the commit messages: agreed and they are rewritten. The transformers macOS job claim was wrong,
and the claim that the earlier note about the trunk path filter was wrong is itself wrong, since the
filter really does not cover this directory and a ciflow tag is a separate push route. The
description already had the correct version of both, and since this repository squashes with the
description, that is the text that lands.

Two I am leaving: the per-test export cost, and the note that the Core ML enable_state and
preserve_sdpa options are read by no exporter. The second is a real product gap this change makes
reachable for the first time, but it is not this change's to fix.

CopilotAI review requested due to automatic review settings August 31, 2026 15:30

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shoumikhin

Copy link
Copy Markdown
ContributorAuthor

Second pass, all four addressed.

The commit messages are actually rewritten now. I said last time they were, and they were not, so
thank you for the specifics. I verified your finding first: at the merge base all six
coreml_fp32_gpu cells of the transformers macOS job passed, including the model my message named, so
the message pointed at a job that was green. That claim, the "no outcome changes" line, the sentence
answering a review with no public existence, and the one calling the trunk path filter note wrong are
all gone. The history is a message-only rewrite, the tree is byte identical.

The tests no longer export a model to check a branch. Routing is decided before the model is
touched, so the preparation is stubbed. Peak memory goes from about 4.3 GB to 420 MB and the three
cases from 7.1 to 3.5 seconds of processor time, with the mutation matrix unchanged.

The KV cache is pinned off, with a line saying why. Your finding was right: with it on the source
transforms import the Qualcomm SDK, so the Core ML with QNN case was avoiding that only through a
default nothing stated.

The two unread Core ML options are a real bug and I filed it separately.enable_state and
preserve_sdpa are parsed, stored, assigned, and read by nothing, and the lfm2 Core ML config shipped
here sets both to true, so that config has been exporting without them silently. It is not this
change's to fix, but this change is what makes it observable, so it should not stay only in a review
thread. Tracked in #22352.

The fix had no test, so nothing stopped the field read returning.
Patching the lowering to raise a marker keeps this off macOS and away from coremltools: what is
asserted is that control reaches the lowering at all, which is exactly what the stale read prevented.
Test plan:
with the fix reaches the Core ML lowering
without it raises AttributeError before any lowering
The first test pinned only one of the two routes the fix repaired, and it stayed green when the whole
exclusion clause was deleted, which silently drops the Vulkan and QNN partitioners. So it guarded the
removed field read and not the routing the clause exists for.
Core ML with QNN is now covered too. It must fall through to the combined lowering, which still
lowers Core ML but keeps the QNN partitioner. That case raised before the fix and it fails on the
mutant, so between them the two tests pin both halves.
Both now use a patched lowering with an assertion on the call, which is how the rest of the tree
writes this, instead of a local marker exception and a stub.
Also removed a paragraph from the test docstring about when the macOS Core ML job runs. A comment
inside a test cannot be checked from the test and goes stale on its own, and the workflow's own path
filter does not cover this directory, so the sentence was misleading either way.
Test plan:
both tests, fixed pass
stale field restored the Core ML case fails with the AttributeError
exclusion clause deleted the QNN case fails
Review found the tests pinned only the QNN half. Dropping just the Vulkan term left both green, and
that mutation sends a Core ML plus Vulkan export to the Core ML lowering, which takes no Vulkan
argument, so the partitioner is dropped in silence. A third case covers it, and the mutation matrix
is now complete:
drop vulkan term the Vulkan case fails
drop qnn term the QNN case fails
whole clause deleted both combined cases fail
stale field restored both Core ML cases fail
The helper also turned Core ML on inside itself, so neither call site showed which backends its case
used, and deleting that hidden line left the test named for Core ML with QNN passing. Backends are
now named at each call.
Three smaller corrections that came with it. The helper resolved the backend field from a string
through getattr, which is the same unchecked lookup this change removes and is against the house
rule on dynamic attribute access. The patch target was built from an f-string, the only one in the
tree, now patch.object. And a bare RuntimeError was the marker, which swallowed real RuntimeErrors
from inside the export and reported only that the lowering was never called; a private marker class
shows the real cause instead.
Both combined cases now also assert the backend flags reach the shared lowering, since a call count
alone cannot see them and the exclusion clause exists to preserve exactly those partitioners.
Each routing case traced and lowered the whole model to check which branch it takes. Stubbing the
model preparation, which the routing decision does not depend on, takes peak memory from about
4.3 GB to 420 MB and the three cases from 7.1 to 3.5 seconds of processor time.
The mutation matrix is unchanged, so the cases still pin what they did before:
drop vulkan term the Vulkan case fails
drop qnn term the QNN case fails
whole clause deleted both combined cases fail
stale field restored both Core ML cases fail
Also pinned the KV cache off. With it on, the source transforms import the Qualcomm SDK, which
routing does not need and which is absent on most machines, so the Core ML with QNN case only avoided
it through a default nothing stated.
CopilotAI review requested due to automatic review settings September 1, 2026 16:52

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@shoumikhin
shoumikhin merged commit 2cd8f26 into pytorch:mainSep 1, 2026
206 of 212 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: examplesChanges to any of our example LLMs integrations, such as Llama3 and Llava

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@shoumikhin@JakeStevens