Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899

Merged
psiddh merged 1 commit into
pytorch:mainfrom
CodeLinaro:pr4b
Aug 28, 2026
Merged

Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom
CodeLinaro:pr4b

Conversation

@qti-horodnic

@qti-horodnicqti-horodnic commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.

What's included

Strategy implementations (2 files + 1 __init__ fix):

  • ExecuTorchModelPreparationStrategy: 5-step flow

    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)

  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter

    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)

  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__

Unit tests:

  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py

PR Review Checklist

  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).

Related PRs

Test plan

Run only tests added in this PR:

python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:

python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:

python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot

pytorch-botBot commented Aug 17, 2026•
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 17, 2026
@qti-horodnic

qti-horodnic commented Aug 17, 2026

Copy link
Copy Markdown
ContributorAuthor

@pytorchbot label "release notes: qualcomm"

@pytorch-botpytorch-botBot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Aug 17, 2026
@psiddh

psiddh commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

@claude review this PR also review this
strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature

quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.

@claude

claudeBot commented Aug 28, 2026•
edited
Loading

Copy link
Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job

PR Review: PR5 — Model preparation & quantization strategies

  • Read default_quantizer_adapter.py + export_utils.make_quantizer
  • Trace the default-path call in ExecuTorchQuantizationStrategy
  • Verify test doubles hide the signature mismatch
  • Review the rest of the PR

1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴

Your read is exactly right. The chain:

  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.

So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.

Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.

Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:

defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)

The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.

2. Same class of bug, one line over: quant_recipe forwarding 🟠

export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.

test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.

3. Endorse the create_autospec recommendation ✅

Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.

Fix all three →

Rest of the PR — looks solid

  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.

The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.
· branch pr4b

@psiddh

psiddh commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)

@qti-horodnic

qti-horodnic commented Aug 28, 2026

Copy link
Copy Markdown
ContributorAuthor

Thanks @psiddh, I've addressed the 2 comments. Should be good to merge

@psiddh
psiddh merged commit 72b2536 into pytorch:mainAug 28, 2026
177 of 180 checks passed
@psiddh

psiddh commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Thanks @psiddh, I've addressed the 2 comments. Should be good to merge

Thanks again for the refactor , it is really looking solid

@qti-horodnicqti-horodnic mentioned this pull request Aug 28, 2026
Merged
psiddh pushed a commit that referenced this pull request Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees

No one assigned

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations - #21899
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Aug 28, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations#21899
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4b
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Summary
This PR implements the model preparation and quantization strategy implementations, replacing the NotImplementedError stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability.
What's included
Strategy implementations (2 files + 1 __init__ fix):
  • ExecuTorchModelPreparationStrategy: 5-step flow
    • load_model → load_tokenizer (via ModelLoaderAdapter)
    • generate_calibration_data (via separately-injectable CalibrationDataAdapter)
    • Optional tokenizer export for on-device runtime
    • Chat template extraction from tokenizer (with extra_options fallback)
    • Validates input config (model_name, soc_model required)
  • ExecuTorchQuantizationStrategy: Full PT2E single-graph pipeline via QuantizerAdapter
    • export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
    • Supports quant_dtype, quant_recipe, and per-channel options via extra_options
    • Handles any Iterable as calibration data (lists, DataLoaders, generators)
    • Validates calibration data is non-empty before export
    • Warns (does not fail) when training_data is provided (QAT deferred)
  • strategies/model_preparation/__init__.py: adds missing ExecuTorchModelPreparationStrategy import to __all__
Unit tests:
  • test_executorch_model_preparation_strategy.py
  • test_executorch_quantization_strategy.py
  • test_default_model_preparation_adapter.py
  • test_default_model_preparation_adapter.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all genai_pipeline tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 17, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21899
Note: Links to docs will display an error until the docs builds have been completed.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 17, 2026
@qti-horodnic
qti-horodnic
commented
Aug 17, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 17, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR also review this

strategies/quantization/default_quantizer_adapter.py — the make_quantizer signature
quant_dtype is a required positional here, but the strategy omits it on the default path expecting export_utils's use_8a8w default — so ExecuTorchQuantizationStrategy() with defaults raises TypeError. Could we give quant_dtype a default here (and create_autospec the adapter in tests)? The bare MagicMock accepts any signature, so this path currently passes CI but would crash in production.
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 6s —— View job
PR Review: PR5 — Model preparation & quantization strategies
  •  Read default_quantizer_adapter.py + export_utils.make_quantizer
  •  Trace the default-path call in ExecuTorchQuantizationStrategy
  •  Verify test doubles hide the signature mismatch
  •  Review the rest of the PR
1. Confirmed: ExecuTorchQuantizationStrategy() with defaults raises TypeError in production 🔴
Your read is exactly right. The chain:
  • default_quantizer_adapter.py:24-30 declares make_quantizer(self, quant_dtype, backend, soc_model, **kwargs) — quant_dtype is a required positional, no default.
  • executorch_quantization_strategy.py:149-161 deliberately omits quant_dtype on the default path (comment: "only pass quant_dtype if explicitly provided, so the adapter's default applies otherwise"), calling make_quantizer(backend=..., soc_model=...).
  • But the adapter has no default to fall back to. The use_8a8w default lives one layer deeper, in export_utils.make_quantizer (export_utils.py:653: quant_dtype: Optional[QuantDtype] = QuantDtype.use_8a8w). The wrapper's required positional shadows it.
So the default path raises TypeError: make_quantizer() missing 1 required positional argument: 'quant_dtype' before it ever reaches export_utils.
Why CI is green:_make_mock_adapter() (test file :28-36) uses a bare MagicMock(), which accepts any signature. test_invoke_passes_correct_args_to_make_quantizer (:158-173) even asserts the call is made withoutquant_dtype — locking in the exact call shape that would crash the real adapter.
Suggested fix — give the wrapper a default that mirrors the intent (let export_utils supply use_8a8w), keeping the lazy-import pattern intact:
defmake_quantizer(
self,
quant_dtype: Any=None,
backend: Any=None,
soc_model: Any=None,
**kwargs: Any,
) ->Any:
...
make_quantizer_kwargs= {"backend": backend, "soc_model": soc_model_str, **kwargs}
ifquant_dtypeisnotNone:
make_quantizer_kwargs["quant_dtype"] =quant_dtypereturn_make_quantizer(**make_quantizer_kwargs)
The QuantizerAdapter Protocol (quantizer_adapter.py:34-40) should get the same = None default so the contract matches.
2. Same class of bug, one line over: quant_recipe forwarding 🟠
export_utils.make_quantizer (export_utils.py:652-665) has no quant_recipe parameter — the signature ends at eps=None. But the strategy adds quant_recipe to make_quantizer_kwargs (:158-159) whenever it's set on the config, and DefaultQuantizerAdapter.make_quantizer forwards it straight through **kwargs to _make_quantizer. On the real adapter that's TypeError: make_quantizer() got an unexpected keyword argument 'quant_recipe'.
test_invoke_passes_quant_recipe_from_config (:192-202) passes only because the mock swallows it. Either quant_recipe needs to be consumed/translated inside the adapter (mapped to submodule_qconfig_list or the per-channel flags?), or it shouldn't be forwarded yet. Worth resolving before this reaches production alongside #1.
3. Endorse the create_autospec recommendation ✅
Both bugs above share one root cause: the tests validate against a mock that accepts any signature. Switching _make_mock_adapter() to create_autospec(DefaultQuantizerAdapter, instance=True) would make the mock enforce the real signature, so a defaults-only call and the quant_recipe kwarg both fail in CI instead of production. Recommend applying it to _make_mock_adapter here and, for consistency, the model-preparation strategy tests too.
Fix all three →
Rest of the PR — looks solid
  • ExecuTorchModelPreparationStrategy (executorch_model_preparation_strategy.py): clean 5-step flow, good separation of the calibration-data adapter from the loader. The "tokenizer chat_template wins over extra_options" precedence (:194-200) and the "example inputs come from the model, never the dataset" invariant (:157-166) are well-reasoned and documented. StageError wrapping with re-raise passthrough is consistent with the quant strategy.
  • DefaultModelLoaderAdapter._select_runtime_tokenizer (:199-222): choosing the runtime tokenizer by filename rather than save_pretrained ordering is the right call, and the docstring explains why. artifacts[-1] fallback + warning is a reasonable last resort.
  • calibrate iterating the raw iterable once with torch.no_grad() and leaving generators/DataLoaders untouched (:101-125) matches the config contract nicely.
  • Validation: calibration_data is None rather than a truthiness test (:211-218) correctly avoids consuming a generator's first sample — good catch in the design.
The two production-crash bugs (#1, #2) are the only blocking items; everything else is well-structured and the docstrings/tests are thorough. Since these are review findings I haven't modified any files — happy to apply the fix above if you'd like.

· branch pr4b
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Accepting to unblock, @qti-horodnic can you take a look at #1 and possibly #2 (its ok if you want to address in upcoming PRs)
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
@psiddh
psiddh
merged commit 72b2536
into
pytorch:mainAug 28, 2026

177 of 180 checks passed

@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
Thanks @psiddh, I've addressed the 2 comments. Should be good to merge
Thanks again for the refactor , it is really looking solid
@qti-horodnicqti-horodnic
mentioned this pull request
Aug 28, 2026
 Merged

psiddh pushed a commit
that referenced
this pull request
Aug 29, 2026
@qti-horodnic
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & infer…
c27baa8
…ence strategy implementations (#22284)
## Summary
This PR implements the __compilation__ and __inference__ strategy
implementations, completing the strategy layer. Each strategy delegates
to injectable adapter interfaces for testability.
### What's included
#### Strategy implementations:
- `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via
`CompilerAdapter`
- Validates model, example_inputs, soc_model, and backend_type are
present
- Passes `example_inputs` explicitly to the adapter (not via
`extra_options`) — mirrors the PR5 fix for the quantization stage
- Delegates to adapter with example_inputs, compile specs, artifact dir,
soc_model, backend_type
- Filters `context.extra_options` to a compilation-relevant allow-list
- Returns artifact paths and optional `ETRecord`
- `ExecuTorchInferenceStrategy`: Runs on-device inference via
`DeviceRunnerAdapter`
- Validates artifact_paths and adapter are present (no default adapter —
device config is required)
- Push → execute → pull results flow
- Two-step protocol: uses `output_data` from execute if present, falls
back to pulled file paths
- Returns inference results, performance metrics, and optional `ETDump`
#### `DefaultCompilerAdapter`:
- `compile_model` signature finalised — mirrors
`to_edge_transform_and_lower_to_qnn`
argument-for-argument, so per-graph lowering inputs (`compile_specs`,
`dep_table`,
`passes_job`, `constant_methods`) are explicit parameters rather than
`extra_options` keys
- Body deliberately raises `NotImplementedError`: the version this
package needs is
the multi-graph one (graph-name-keyed dicts, single multi-method `.pte`
for weight
sharing), so it lands with the strategy-level fan-out that calls it
rather than
being written single-graph and then replaced. Inject a custom
`CompilerAdapter` for now.
#### Config addition:
- `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` —
sourced from the model via `ModelLoaderAdapter.get_example_inputs`
#### Orchestrator wiring:
- `genai_pipeline.py` `_run_compilation`: passes
`example_inputs=model_prep_output.example_inputs`
#### Unit tests:
- `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py`
- `test_compilation_input_config.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
#20409
- PR 2: Strategy interfaces & stage wrappers:
#20795
- PR 3: Pipeline orchestrator:
#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
#21751
- PR 5: Model preparation & quantization strategies:
#21899
- PR 6: Compilation & inference strategy implementations: this pr.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%
```
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh