Arm backend: Make composable_quantizer default - #19758

Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom
AdrianLundell:change-1253855
Jul 16, 2026
Merged

Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom
AdrianLundell:change-1253855

Conversation

@AdrianLundell

@AdrianLundellAdrianLundell commented May 25, 2026•
edited
Loading

Copy link
Copy Markdown
Collaborator

A few fixes needed:

  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.

cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani

@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell requested review from 3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell added help wanted Extra attention is needed partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk release notes: arm Changes to the ARM backend delegate labels May 25, 2026
@pytorch-bot

pytorch-botBot commented May 25, 2026•
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 130f60b with merge base e182e8b (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 25, 2026
@linux-foundation-easycla

linux-foundation-easyclaBot commented May 25, 2026•
edited
Loading

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@github-actionsgithub-actionsBot added the module: arm Issues related to arm backend label May 25, 2026
CopilotAI reviewed May 25, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).

Changes:

  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync

meta-codesyncBot commented May 26, 2026

Copy link
Copy Markdown
Contributor

@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.

@rascani

rascani commented May 26, 2026

Copy link
Copy Markdown
Contributor

Kicking off an internal test run.

@rascani

rascani commented May 26, 2026

Copy link
Copy Markdown
Contributor

It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.

https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121

@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI review requested due to automatic review settings May 27, 2026 08:45
CopilotAI reviewed May 27, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py
Comment on lines +308 to +321
elif node.target in _fixed_input_qspec_ops:

input_act_qspec = super().get_input_act_qspec(node, input_node)
num_bits = torch.iinfo(input_act_qspec.dtype).bits
qparams = _fixed_input_qspec_ops[node.target][num_bits]
return FixedQParamsQuantizationSpec(
dtype=input_act_qspec.dtype,
scale=qparams.scale,
zero_point=qparams.zero_point,
quant_min=input_act_qspec.quant_min,
quant_max=input_act_qspec.quant_max,
qscheme=input_act_qspec.qscheme,
is_dynamic=input_act_qspec.is_dynamic,
)
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md Outdated
Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md Outdated
@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani

rascani commented May 27, 2026

Copy link
Copy Markdown
Contributor

Kicking off another internal run.

@digantdesai

digantdesai commented May 27, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

FYI @3l1 - not sure if you saw this.

@digantdesai

digantdesai commented May 27, 2026

Copy link
Copy Markdown
Contributor

From @rascani 's internal CI runs,

executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.

@rascani

rascani commented May 27, 2026

Copy link
Copy Markdown
Contributor

From @rascani 's internal CI runs,

executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.

I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.

@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell

AdrianLundell commented Jun 16, 2026

Copy link
Copy Markdown
CollaboratorAuthor

I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?

@AdrianLundell

AdrianLundell commented Jun 16, 2026

Copy link
Copy Markdown
CollaboratorAuthor

I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?

@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI review requested due to automatic review settings June 17, 2026 13:27
CopilotAI reviewed Jun 17, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py Outdated
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell

AdrianLundell commented Jun 29, 2026

Copy link
Copy Markdown
CollaboratorAuthor

I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.

@AdrianLundell

AdrianLundell commented Jun 30, 2026

Copy link
Copy Markdown
CollaboratorAuthor

Look like the quantizers have diverged again, I will fix in a separate commit.

@digantdesai

digantdesai commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Still blocked on us, sorry.

digantdesai
digantdesai approved these changes Jul 9, 2026
View reviewed changes

@digantdesaidigantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell merged commit 432353a into pytorch:mainJul 16, 2026
504 of 506 checks passed
@zingo

zingo commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Yeah!

@digantdesai

digantdesai commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review automatically exported from Phabricator review in Meta.

sorry error on my part. Let me revert.

@digantdesaidigantdesai mentioned this pull request Jul 16, 2026
Merged
@AdrianLundell

AdrianLundell commented Jul 17, 2026

Copy link
Copy Markdown
CollaboratorAuthor

I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai

rascani pushed a commit that referenced this pull request Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani

rascani commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai

We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.

@rascanirascani mentioned this pull request Jul 17, 2026
Merged
@rascani

rascani commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai

We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.

🤞 I think we should be good now.

@vacu9708vacu9708 mentioned this pull request Jul 28, 2026
Merged
Erik-Lundell pushed a commit that referenced this pull request Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees

No one assigned

Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Arm backend: Make composable_quantizer default - #19758
Merged
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Jul 16, 2026
Merged
Arm backend: Make composable_quantizer default#19758
AdrianLundell merged 14 commits into
pytorch:mainfrom 
AdrianLundell:change-1253855
Conversation
@AdrianLundell
@AdrianLundellAdrianLundell
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
A few fixes needed:
  • Add new ops added since initial upstream of composable_quantizer
  • Add while-op quantize fix from 3be4546 to TosaQuantizerV2
  • Add fixed_qparams fix from fb90480 to TosaQuantizerV2
  • Update some tests to mirror new behaviours
  • Update quanitzer_tutorial to not be WIP
  • Remove hardsigmoid from FUSED_ACTIVATION_OPS
  • Explicitly check that weights and biases are input args to conv/ linear ops. The assumption that wights and biases are the only parameters of networks does not hold for real models.
cc @digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@AdrianLundell
Arm backend: Make composable_quantizer default
4595733
A few fixes needed:
- Add new ops added since initial upstream of composable_quantizer
- Add while-op quantize fix from 3be4546 to TosaQuantizerV2
- Add fixed_qparams fix from fb90480 to TosaQuantizerV2
- Update some tests to mirror new behaviours
- Update quanitzer_tutorial to not be WIP
- Remove hardswish from FUSED_ACTIVATION_OPS
- Explicitly check that weights and biases are input args to
conv/ linear ops. The assumption that wights and biases are
the only parameters of networks does not hold for real models.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ifa127a73d4db45cd2d3461101f97c0cf852bf7bf
@AdrianLundell
AdrianLundell
requested review from
3l1 and CopilotMay 25, 2026 15:05
@AdrianLundellAdrianLundell
added

help wanted
Extra attention is needed
partner: arm
For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm
ciflow/trunk

release notes: arm
Changes to the ARM backend delegate
labels
May 25, 2026
@pytorch-bot
pytorch-botBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19758
Note: Links to docs will display an error until the docs builds have been completed.
✅ No Failures
As of commit 130f60b with merge base e182e8b (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
May 25, 2026
@linux-foundation-easycla
linux-foundation-easyclaBot
commented
May 25, 2026•

edited

Loading

Copy link


Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.
@github-actionsgithub-actionsBot
added
the 
module: arm
Issues related to arm backend
label
May 25, 2026
CopilotAI
reviewed
May 25, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
This PR switches the Arm backend’s TOSAQuantizer to use the composable quantizer implementation by default, and updates Arm quantization annotation/support logic plus tests/tutorial materials to match the new behaviors (including while-loop and fixed-qparams handling).
Changes:
  • Make TOSAQuantizer default to use_composable_quantizer=True.
  • Extend/update quantizer support + annotation behavior (e.g., while-loop shared-qspec handling, fixed-qparams input qspecs for trig ops, additional supported ops).
  • Update Arm backend tests and the Arm quantizer tutorial notebook to reflect the new defaults/behaviors.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
FileDescription
examples/arm/quantizer_tutorial.ipynbRemoves WIP framing in the composable quantizer tutorial intro.
backends/arm/test/ops/test_while.pyAdds an INT while-loop test path forcing composable quantizer usage.
backends/arm/test/ops/test_transpose_conv2d.pyUpdates tests to use TOSAQuantizationConfig for global config setup.
backends/arm/test/ops/test_to_copy.pySimplifies redundant-cast xfail configuration shared between FP/INT.
backends/arm/test/misc/test_shared_qspecs.pyUpdates golden expectations for shared-qspec annotation counts/qparams.
backends/arm/test/misc/test_quant_custom_meta.pyAdjusts test quantizer config (including set_io(None)) to match new behavior.
backends/arm/quantizer/quantizer_support.pyUpdates supported/fused patterns (and adds more supported ops).
backends/arm/quantizer/quantization_config.pyAdds fixed-qparams input spec generation for specific trig ops under composable flow.
backends/arm/quantizer/arm_quantizer.pyMakes composable quantizer the default for TOSAQuantizer.
backends/arm/quantizer/arm_quantizer_utils.pyTightens weight/bias identification and adds while-loop shared-qspec special-casing; extends shared-qspec op list.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment threadbackends/arm/quantizer/arm_quantizer.py
Comment threadbackends/arm/quantizer/quantizer_support.py
@AdrianLundell
Make vgf/ethosu-quantizers composable by default
c2a9306
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I75e207d010b2bdc9abc86023153c86d2c96af3fd
@meta-codesync
meta-codesyncBot
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
@rascani has imported this pull request. If you are a Meta employee, you can view this in D106381887.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
Kicking off an internal test run.
@rascani
rascani
commented
May 26, 2026

Copy link


Copy Markdown

Contributor
It looks like the new quantizer breaks examples/models/llama/tests/test_export_llama_lib.py, because of asserts on quantizer.module_type_config, which is not implemented by the new quantizer. I think we should probably just remove those asserts in the tests unless you want to expose the module type config in the new quantizer.
https://github.com/pytorch/executorch/blob/main/examples/models/llama/tests/test_export_llama_lib.py#L121
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
af8056f
…tream/change-1253855
Change-Id: I86bc8025e41175a796fa0ff7aabc9847b1de923a
@AdrianLundell
Remove quanitzer config asserts from test_export_llama_lib.py
098d79b
These configs does not exist in the new quantizer, so checking it
does not make sense anymore.
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ic6c0b303466010be59e9c9c37fb179938c412a16
CopilotAI
review requested due to automatic review settings
May 27, 2026 08:45
CopilotAI
reviewed
May 27, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Comment threadbackends/arm/quantizer/quantization_config.py

Comment on lines

+308
 to 
+321

 elif node.target in _fixed_input_qspec_ops:

 input_act_qspec = super().get_input_act_qspec(node, input_node)
 num_bits = torch.iinfo(input_act_qspec.dtype).bits
 qparams = _fixed_input_qspec_ops[node.target][num_bits]
 return FixedQParamsQuantizationSpec(
 dtype=input_act_qspec.dtype,
 scale=qparams.scale,
 zero_point=qparams.zero_point,
 quant_min=input_act_qspec.quant_min,
 quant_max=input_act_qspec.quant_max,
 qscheme=input_act_qspec.qscheme,
 is_dynamic=input_act_qspec.is_dynamic,
 )
Comment threaddocs/source/backends/arm-vgf/arm-vgf-quantization.md
Outdated

Comment threaddocs/source/backends/arm-ethos-u/arm-ethos-u-quantization.md
Outdated

@AdrianLundell
Arm backend: Fix review comments
155b4df
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I7760780d4f06e300f17575e7c6cb34c6dfdae64c
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
Kicking off another internal run.
@digantdesai
digantdesai
commented
May 27, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
FYI @3l1 - not sure if you saw this.
@digantdesai
digantdesai
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
@rascani
rascani
commented
May 27, 2026

Copy link


Copy Markdown

Contributor
From @rascani 's internal CI runs,
executorch/backends/arm/quantizer/arm_quantizer.py", line 522, in global_config
raise NotImplementedError(
NotImplementedError: Composable quantizer does not allow setting global_config directly. Please use set_global() instead.

These should be forward fixed.
I already have these fixed in D106539874. I'll land it ahead of time so the default switch can be a no-op.
@AdrianLundell
Add new ops to test_portable_int8
6c4194e
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: Ib6fefa3ba308d955b46e23095633f54df8ed3bf0
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I would recommend replacing accessing e.g. module_name_config directly with using the official set_module_name setter API in that case, does that resolve the issue?
@AdrianLundell
AdrianLundell
commented
Jun 16, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I realize we could also make a setter which takes the dict and calls set_module_name on all entries to have no breakage, would that help?
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
d84d3e9
…tream/change-1253855
Change-Id: I2c920401d797530b7bab6189941636e1b2035de1
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
da02dcc
…tream/change-1253855
Change-Id: I931de7467e5697c3a15420913e4b68a8e088a8b4
@AdrianLundell
Arm backend: Back-port setting quantization configs from dicts
6cb6398
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I9c8dc4e9690d591a286ad22376c509babfb24c0e
CopilotAI
review requested due to automatic review settings
June 17, 2026 13:27
CopilotAI
reviewed
Jun 17, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
Outdated

Comment threadbackends/arm/quantizer/arm_quantizer_utils.py
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
eeb312c
…tream/change-1253855
Change-Id: I719898b5ea2056d5ab8e9723fb41f612de8e04e1
@AdrianLundell
Arm backend: Fix merge issue + add ops to quantizer support-dict
f6c7efc
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I39fd172f326909a084c8dc87613ed02d2032963d
@AdrianLundell
AdrianLundell
commented
Jun 29, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I had to take a break from this one, but now you should be able to set quantization specs from dicts as you are used to @rascani.
@AdrianLundell
AdrianLundell
commented
Jun 30, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Look like the quantizers have diverged again, I will fix in a separate commit.
@digantdesai
digantdesai
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Still blocked on us, sorry.
digantdesai
digantdesai
approved these changes
Jul 9, 2026
View reviewed changes
@digantdesaidigantdesai
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.
@AdrianLundell
Merge branch 'main' ofhttps://github.com/pytorch/executorchinto ups…
2315aa2
…tream/quantizer
Change-Id: Ieb92342ab40135764270031d19e80a752595d96d
@AdrianLundell
Arm backend: Fix CI failures
130f60b
- Annotate IO with none for non supported nonzero operator
- Ensure while cast node is created with a unique meta dict
to avoid corrupted quantization annotations
Signed-off-by: Adrian Lundell <adrian.lundell@arm.com>
Change-Id: I22e4759cef7cff79f6dce26f147012f01d0d4883
@AdrianLundell
AdrianLundell
merged commit 432353a
into
pytorch:mainJul 16, 2026

504 of 506 checks passed

@zingo
zingo
commented
Jul 16, 2026

Copy link


Copy Markdown

Collaborator
Yeah!
@digantdesai
digantdesai
commented
Jul 16, 2026

Copy link


Copy Markdown

Contributor
Review automatically exported from Phabricator review in Meta.
sorry error on my part. Let me revert.
@digantdesaidigantdesai
mentioned this pull request
Jul 16, 2026
 Merged

@AdrianLundell
AdrianLundell
commented
Jul 17, 2026

Copy link


Copy Markdown

CollaboratorAuthor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
rascani pushed a commit
that referenced
this pull request
Jul 17, 2026
@digantdesai
Revert "Arm backend: Make composable_quantizer default" (#20982)
b9af901
Reverts #19758
cc @freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
@rascanirascani
mentioned this pull request
Jul 17, 2026
 Merged

@rascani
rascani
commented
Jul 17, 2026

Copy link


Copy Markdown

Contributor
I see, when would you say this will be unblocked and can I make any updates to make it easier? @digantdesai
We had a bit of a race condition with an internal change. Barring no surprises, I am hoping to reland this today.
🤞 I think we should be good now.
@vacu9708vacu9708
mentioned this pull request
Jul 28, 2026
 Merged

Erik-Lundell pushed a commit
that referenced
this pull request
Jul 31, 2026
@vacu9708
Arm backend: Give SiLU its own output quantization params (#21437)
761c64c
### Summary
SiLU's quantized output is forced to reuse its input's quantization
scale, which wastes resolution for no benefit.
This commit gives SiLU its own output quantization spec.
### Where this happens:
| Quantizer | Mechanism | `silu.default` | `silu_.default` |
| --- | --- | --- | --- |
| Composable (`_TOSAQuantizerV2`, default since #19758) |
`TOSAQuantizationConfig.<br>SHARED_OUTPUT_ACT_QSPEC_PATTERNS` | shared
(bug) | shared (bug) |
| Legacy (`_TOSAQuantizerV1`) | `quantization_annotator.py` op sets |
own qspec | shared (bug) |
### Why quantization sharing is wrong for SiLU:
<img width="512" height="492" alt="image"
src="https://github.com/user-attachments/assets/ea71702f-bbe0-402a-b2b0-595b25e0d170"
/>
- **SiLU non-linearly compresses and warps its input range:**
- Unlike ReLU, which maintains a 1:1 linear mapping ($y = x$) for
non-negative inputs, SiLU is non-linear across its entire domain.
- For negative inputs, its output never goes below -0.278, no matter how
negative the input gets. So its output range is always narrower than its
input's. Reusing the input scale misaligns the quantization bins with
the actual output distribution, **wasting resolution.**
- **Reusing the input's scale doesn't save any computation at runtime:**
SiLU runs as a lookup table (`class TableOps`) whose entries are
precomputed at compile time, not runtime.
### Changes
| File | Change | Why |
| --- | --- | --- |
| `quantizer/quantization_config.py` | Remove
`silu.default`/`silu_.default` from `SHARED_OUTPUT_ACT_QSPEC_PATTERNS` |
The accuracy loss as described above. |
| `quantizer/quantization_annotator.py` | Move `silu_.default` into
`_one_to_one`, next to `silu.default` | The legacy annotator classified
the two variants differently. #17202 put `silu.default` in `_one_to_one`
(own qspec) but `silu_.default` in
`_one_to_one_shared_input_or_input_act_qspec` (shared qspec). |
| `test/ops/test_silu.py` | `Silu.forward` clones its input | Latent bug
found while working on this change: SiLU(inplace=True) was mutating the
test pipeline's shared input tensor which the pipeline reuses. It stayed
invisible because the buggy (too large) output scale made the comparison
helper(`compare_rel_frobenius_and_cosine_similarity()`) treat every
value as noise and skip the check entirely, so the test kept passing
without validating anything. |
| `test/misc/test_shared_qspecs.py` | Add `NonSharedQspecSilu` +
`test_silu_does_not_share_input_qspec` | Check the emitted q/dq
parameters, the same way the rest of `test_shared_qspecs.py` checks
sharing. |
### Testing
```bash
pytest backends/arm/test/misc/test_shared_qspecs.py
# 17 passed
pytest backends/arm/test/ops/test_silu.py -k "not vgf"
# 34 passed, 32 xfailed
```
cc @kimishpatel@jerryzh168@digantdesai@freddan80@per@zingo@oscarandersson8218@mansnils@Sebastian-Larsson@robell@rascani
Signed-off-by: Youngsik Yang <vacu9708@gmail.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort, defaults to Lite
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@digantdesaidigantdesaidigantdesai approved these changes
@3l13l1Awaiting requested review from 3l1
@mergennachinmergennachinAwaiting requested review from mergennachinmergennachin is a code owner
@rascanirascaniAwaiting requested review from rascani

Assignees


No one assigned


Labels

ciflow/trunkCLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.help wantedExtra attention is neededmodule: armIssues related to arm backendpartner: armFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Armrelease notes: armChanges to the ARM backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

5 participants

@AdrianLundell@rascani@digantdesai@zingo