Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces, default implementations & dataset providers by qti-horodnic · Pull Request #21751 · pytorch/executorch

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic · 2026-08-11T19:54:42Z

Summary

This PR adds the adapter layer that wraps external APIs (ExecuTorch, QNN SDK, HuggingFace) behind injectable Protocol interfaces for testability. Each strategy implementation (in subsequent PRs) delegates to these adapters rather than calling external APIs directly, enabling unit tests with mocked dependencies.

What's included

Adapter Protocols (6 files):

QuantizerAdapter: Protocol wrapping make_quantizer, prepare_pt2e, calibrate, convert_pt2e
CompilerAdapter: Protocol wrapping ExportSession compilation flow + CompilationResult dataclass
DeviceRunnerAdapter: Protocol wrapping SimpleADB push/execute/pull + InferenceResult dataclass
ModelLoaderAdapter: Protocol wrapping HuggingFace model/tokenizer loading
CalibrationDataAdapter: Protocol for calibration dataset construction
TrainingDataAdapter: Protocol for QAT training data (yields (features, labels) pairs)

Default Implementations (6 files):

DefaultQuantizerAdapter: Delegates to export_utils.make_quantizer + torchao.quantization.pt2e
DefaultCompilerAdapter: Placeholder for recipe-based compilation (depends on ExportRecipe/ExportSession APIs not yet available). Raises NotImplementedError with guidance to inject a custom CompilerAdapter using to_edge_transform_and_lower_to_qnn.
DefaultDeviceRunnerAdapter: Delegates to SimpleADB for on-device execution
DefaultModelLoaderAdapter: Delegates to HuggingFace AutoModelForCausalLM + AutoTokenizer
DefaultCalibrationDataAdapter: Random token sequences; accepts a caller-supplied dataset or DataLoader via extra_options["dataset"]
DefaultTrainingDataAdapter: Pass-through for caller-supplied QAT data; raises ValueError if absent (labelled data can't be synthesized)

Configuration:

.coveragerc updated to omit default_*_adapter.py files (integration-test-only, require real SDK/hardware)
__init__.py files updated to export new adapter types

New `datasets/` package

Dataset providers are a cross-stage concern, not a model-preparation detail — the same corpus feeds PTQ calibration during quantization and
on-device result evaluation during inference (including pre-built .pte flows where model preparation never runs).
They therefore live in a top-level datasets/ package rather than under strategies/model_preparation/.

PR Review Checklist

All new classes follow single responsibility (one class per file) - Yes.
All dependencies are injected via constructor with sensible defaults - Yes.
All external calls are behind injectable interfaces - Yes (Protocol pattern).
Unit tests cover every public method - Yes.
No existing files are modified (Phase 1 constraint) - Yes (only existing files modified are those added in previous GenAI prs).
Docstrings on all public classes and methods - Yes.
Type annotations on all function signatures - Yes.
Logging follows the strategy in the LLD - Yes (lazy imports, debug-level logging in defaults).

Related PRs

PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
PR 4: Adapter interfaces, default implementations & dataset providers: this pr.
PR 5: Model preparation & quantization strategies: pending
PR 6: Compilation & inference strategy implementations: pending.
PR 7: Integration & E2E tests: pending.

Test plan

python -m pytest \
backends/qualcomm/genai_pipeline/tests/ \
-v

All existing tests continue to pass (no regressions).

Test Coverage

Command to run:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 7 1 0 0 86% 40
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 475 10 44 1 98% 417 10 30 1 98%

pytorch-bot · 2026-08-11T19:54:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21751

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 3aac96f with merge base c56e6bf ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

qti-horodnic · 2026-08-11T19:54:56Z

@pytorchbot label "release notes: qualcomm"

psiddh · 2026-08-15T01:10:30Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external
dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

psiddh · 2026-08-15T01:28:01Z

@qti-horodnic Firstly, this is a great refactor, modular, composable, and creating abstractions for all external dependencies. This will also help a lot with unit testing things. In general +1 to this direction. I have a few high-level thoughts, nothing blocking, mostly about what gets frozen here vs. what lands in PRs 5-7.

Per-model transforms and the registry. The loader docstring says per-model graph/weight transforms will be "declared as data on the model registry entry" rather than via adapter subclasses, which is great imo. One thing that might be worth doing before the interfaces settle: today those transforms aren't referenceable yet — convert_linear_to_conv2d is a shared function, but things like the Gemma RMSNorm +1 offset and the partial-RoPE permute are inline in _prepare_model. Could we extract them into named transforms first, then add the transform-list column? Also worth noting one of them is the HF-module to static flat-KV-module swap, which means a transform is Module -> Module rather than an in-place mutation, that may constrain the signature.
Related: LLMModelConfig / @register_llm_model in examples/qualcomm/oss_scripts/llama/ is already a registry (carries repo_id, quant_recipe, num_sharding, convert_weights). Is the plan to extend that one, or introduce a new one? Just want to avoid ending up with two sources of per-model truth.
No "graph set" concept : One weight set can expand into N graphs, and N is a strategy decision: static_llama yields
prefill / decode / token-embedding / calibrate-only; a single multi-token graph collapses to one. Every adapter here assumes N=1 structurally, and the fan-out , plus the cross-graph encoding copy that's required whenever N>1 — is punted to strategies that don't exist yet, so each will re-derive the same set. A graph-set type (like build_qnn_llm_graphset) is the missing abstraction; N=1 is just its degenerate case. Wdyt ?
Are we replacing llama.py with this new GenAI pipeline, or will both paths exist?
DefaultCompilerAdapter. Small one, the default currently raises NotImplementedError and the working path is "inject your own." Would it make sense to have the proven to_edge_transform_and_lower_to_qnn path be the default (or) following PRs will fill it ?

qti-horodnic · 2026-08-16T00:06:07Z

@psiddh
Thanks for taking the time to review and give feedback, I appreciate it. I'll address each one of your comments in a follow-up comment below, but before that let me just provide a brief overview of the project's structure.

Phase 1: The current work, includes PRs 1-7 as outlined in the PR description. Includes only the addition of the skeleton of the new GenAI infrastructure. No existing code paths change behavior, and the skeleton isn't wired into any current entry point. This phase is purely additive and inert by construction, which is why several interfaces here are single-graph / stub-bodied: they're the N=1 degenerate case, with the general form landing in Phase 2.

Phase 2: The next phase will include 4 PRs, divided between me and @DannyYuyang-quic into 2 (roughly) parallel work streams. This phase will include moving legacy code (e.g. llama.py) behind the adapters with a compat shim. The bulk of the logic implementation happens in this phase: multi-graph export, encoding reconciliation, the working compiler adapter, and the device runner.
Note that the existing code will stay exactly where it is in this phase, no code is removed, in order to maintain backwards compatibility.

Phase 3: Cleanup of old code. This will be done after only phase 2 has been completed and all critical code paths have been validated successfully. Nothing is deleted before this phase.

qti-horodnic · 2026-08-16T00:44:18Z

Agreed. I looked at _prepare_model more closely and there are actually four kinds: state_dict mutations pre-load, one that needs the constructed module (the RoPE permute reads head counts), in-place module mutations, and Module -> Module. Ordering between them is load bearing, so it'll be two lists rather than a flat transforms column. Extracting them as named functions in Phase 2, before anything depends on the shape.

On the HF -> static flat-KV swap: I'd keep that out of the transform list since it's "which class to construct" and is already registry data. Separating the two keeps the Module -> Module constraint from spreading to everything else.

I plan to extend LLMModelConfig. This will be done as part of phase 2 as well.
Agreed, and it's already the frozen interface between the phase 2 work streams.
As mentioned in my comment above, llama.py stays as is through phase 2. It will be removed in phase 3, gated on parity testing the functionality.
Good catch, the docstring is inaccurate. The real implementation against those APIs is in PR6 in this phase, so the default is functional there rather than "inject your own". Fixed the docstring here.

…and default implementations

psiddh · 2026-08-17T07:19:24Z

Merging it now (inert for now) , as it unblocks the next few PRs

… quantization strategy implementations (#21899) ## Summary This PR implements the __model preparation__ and __quantization__ strategy implementations, replacing the `NotImplementedError` stubs with real logic. Each strategy delegates to injectable adapter interfaces (from PR4) for testability. ### What's included #### Strategy implementations (2 files + 1 `__init__` fix): - `ExecuTorchModelPreparationStrategy`: 5-step flow - `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`) - `generate_calibration_data` (via separately-injectable `CalibrationDataAdapter`) - Optional tokenizer export for on-device runtime - Chat template extraction from tokenizer (with `extra_options` fallback) - Validates input config (`model_name`, `soc_model` required) - `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via `QuantizerAdapter` - export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e - Supports `quant_dtype`, `quant_recipe`, and per-channel options via `extra_options` - Handles any `Iterable` as calibration data (lists, DataLoaders, generators) - Validates calibration data is non-empty before export - Warns (does not fail) when `training_data` is provided (QAT deferred) - `strategies/model_preparation/__init__.py`: adds missing `ExecuTorchModelPreparationStrategy` import to `__all__` #### Unit tests: - `test_executorch_model_preparation_strategy.py` - `test_executorch_quantization_strategy.py` - `test_default_model_preparation_adapter.py` - `test_default_model_preparation_adapter.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: this pr. - PR 6: Compilation & inference strategy implementations: pending. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \ backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \ --cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \ --cov=backends/qualcomm/genai_pipeline/strategies/quantization \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 161 0 32 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all `genai_pipeline` tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 593 9 76 1 99% ```

…ence strategy implementations (#22284) ## Summary This PR implements the __compilation__ and __inference__ strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability. ### What's included #### Strategy implementations: - `ExecuTorchCompilationStrategy`: Compiles model to .pte artifacts via `CompilerAdapter` - Validates model, example_inputs, soc_model, and backend_type are present - Passes `example_inputs` explicitly to the adapter (not via `extra_options`) — mirrors the PR5 fix for the quantization stage - Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type - Filters `context.extra_options` to a compilation-relevant allow-list - Returns artifact paths and optional `ETRecord` - `ExecuTorchInferenceStrategy`: Runs on-device inference via `DeviceRunnerAdapter` - Validates artifact_paths and adapter are present (no default adapter — device config is required) - Push → execute → pull results flow - Two-step protocol: uses `output_data` from execute if present, falls back to pulled file paths - Returns inference results, performance metrics, and optional `ETDump` #### `DefaultCompilerAdapter`: - `compile_model` signature finalised — mirrors `to_edge_transform_and_lower_to_qnn` argument-for-argument, so per-graph lowering inputs (`compile_specs`, `dep_table`, `passes_job`, `constant_methods`) are explicit parameters rather than `extra_options` keys - Body deliberately raises `NotImplementedError`: the version this package needs is the multi-graph one (graph-name-keyed dicts, single multi-method `.pte` for weight sharing), so it lands with the strategy-level fan-out that calls it rather than being written single-graph and then replaced. Inject a custom `CompilerAdapter` for now. #### Config addition: - `CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]]` — sourced from the model via `ModelLoaderAdapter.get_example_inputs` #### Orchestrator wiring: - `genai_pipeline.py` `_run_compilation`: passes `example_inputs=model_prep_output.example_inputs` #### Unit tests: - `test_executorch_compilation_strategy.py` - `test_executorch_inference_strategy.py` - `test_compilation_input_config.py` ### PR Review Checklist - All new classes follow single responsibility (one class per file) - Yes. - All dependencies are injected via constructor with sensible defaults - Yes. - All external calls are behind injectable interfaces - Yes (adapter pattern). - Unit tests cover every public method - Yes (100% coverage on strategy impls). - Docstrings on all public classes and methods - Yes. - Type annotations on all function signatures - Yes. - Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step). ### Related PRs - PR 1: Core data model, engine routing & exceptions: #20409 - PR 2: Strategy interfaces & stage wrappers: #20795 - PR 3: Pipeline orchestrator: #21149 - PR 4: Adapter interfaces, default implementations & dataset providers: #21751 - PR 5: Model preparation & quantization strategies: #21899 - PR 6: Compilation & inference strategy implementations: this pr. - PR 7: Integration & E2E tests: pending. ## Test plan ### Run only tests added in this PR: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py -v ``` ### Run only this PR's tests with coverage: ``` python -m pytest \ backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \ backends/qualcomm/genai_pipeline/tests/strategies/inference/ \ backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \ --cov=backends/qualcomm/genai_pipeline/strategies/compilation \ --cov=backends/qualcomm/genai_pipeline/strategies/inference \ --cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------- TOTAL 127 0 16 0 100% ``` ### Run all `genai_pipeline` tests: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ -v ``` ### Run all tests with coverage: ``` python -m pytest backends/qualcomm/genai_pipeline/tests/ \ --cov=backends/qualcomm/genai_pipeline \ --cov-config=backends/qualcomm/.coveragerc \ --cov-report=term-missing ``` Result: ``` Name Stmts Miss Branch BrPart Cover Missing ---------------------------------------------------------------------------------------------------------------------------------------------------- backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100% backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100% backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100% backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100% backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100% backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201 backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100% backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100% backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37 backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100% backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100% backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100% backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100% ---------------------------------------------------------------------------------------------------------------------------------------------------- TOTAL 668 9 92 1 99% ```

qti-horodnic requested review from abhinaykukkadapu and psiddh as code owners August 11, 2026 19:54

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 11, 2026

pytorch-botBot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Aug 11, 2026

Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces …
3aac96f
…and default implementations

qti-horodnic force-pushed the pr4 branch from 45dafba to 3aac96fCompare August 16, 2026 00:54

psiddh approved these changes Aug 17, 2026
View reviewed changes

psiddh reviewed Aug 17, 2026
View reviewed changes

Comment threadbackends/qualcomm/genai_pipeline/strategies/model_preparation/default_model_loader_adapter.py

psiddh merged commit 48974c2 into pytorch:mainAug 17, 2026
184 checks passed

qti-horodnic mentioned this pull request Aug 17, 2026
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations #21899
Merged

qti-horodnic mentioned this pull request Aug 28, 2026
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations #22284
Merged