Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284

Merged
psiddh merged 1 commit into
pytorch:mainfrom
CodeLinaro:pr4c
Aug 29, 2026
Merged

Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom
CodeLinaro:pr4c

Conversation

@qti-horodnic

@qti-horodnicqti-horodnic commented Aug 28, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.

What's included

Strategy implementations:

  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter

    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord

  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter

    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump

DefaultCompilerAdapter:

  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    sharing), so it lands with the strategy-level fan-out that calls it rather than
    being written single-graph and then replaced. Inject a custom CompilerAdapter for now.

Config addition:

  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs

Orchestrator wiring:

  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs

Unit tests:

  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py

PR Review Checklist

  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).

Related PRs

Test plan

Run only tests added in this PR:

python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:

python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:

python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:

Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot

pytorch-botBot commented Aug 28, 2026•
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 5 Pending

As of commit df39537 with merge base 7f4b8ad (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-clameta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 28, 2026
@qti-horodnic

qti-horodnic commented Aug 28, 2026

Copy link
Copy Markdown
ContributorAuthor

@pytorchbot label "release notes: qualcomm"

@pytorch-botpytorch-botBot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Aug 28, 2026
@psiddh

psiddh commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

@claude review this PR

@claude

claudeBot commented Aug 28, 2026•
edited
Loading

Copy link
Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job

PR Review: Compilation & Inference Strategy Implementations

  • Read changed files
  • Review compilation strategy
  • Review inference strategy
  • Review adapters & config
  • Post review feedback

Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.

1. Auto-wired pipeline can never run compilation or inference (design gap)

The orchestrator instantiates strategies with no arguments:

# genai_pipeline.py:163returnstage_cls(strategy_cls())

But both strategies added here depend on an injected adapter that has no working default:

  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.

So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.

I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.

2. Input data is never pushed to the device

DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:

# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)

InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.

3. inference_results type vs. output_data contents

# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]

InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.

4. example_inputs presence check vs. empty tuple

_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."

Minor / nits

  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.

Tests

Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).

Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.
· branch pr4c

@qti-horodnic

qti-horodnic commented Aug 28, 2026

Copy link
Copy Markdown
ContributorAuthor

Addressing Claude's comments:

  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.

@psiddh
psiddh merged commit c27baa8 into pytorch:mainAug 29, 2026
190 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees

No one assigned

Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations - #22284
Merged
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Aug 29, 2026
Merged
Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations#22284
psiddh merged 1 commit into
pytorch:mainfrom 
CodeLinaro:pr4c
Conversation
@qti-horodnic
@qti-horodnicqti-horodnic
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
This PR implements the compilation and inference strategy implementations, completing the strategy layer. Each strategy delegates to injectable adapter interfaces for testability.
What's included
Strategy implementations:
  • ExecuTorchCompilationStrategy: Compiles model to .pte artifacts via CompilerAdapter
    • Validates model, example_inputs, soc_model, and backend_type are present
    • Passes example_inputs explicitly to the adapter (not via extra_options) — mirrors the PR5 fix for the quantization stage
    • Delegates to adapter with example_inputs, compile specs, artifact dir, soc_model, backend_type
    • Filters context.extra_options to a compilation-relevant allow-list
    • Returns artifact paths and optional ETRecord
  • ExecuTorchInferenceStrategy: Runs on-device inference via DeviceRunnerAdapter
    • Validates artifact_paths and adapter are present (no default adapter — device config is required)
    • Push → execute → pull results flow
    • Two-step protocol: uses output_data from execute if present, falls back to pulled file paths
    • Returns inference results, performance metrics, and optional ETDump
DefaultCompilerAdapter:
  • compile_model signature finalised — mirrors to_edge_transform_and_lower_to_qnn
    
argument-for-argument, so per-graph lowering inputs (compile_specs, dep_table,
    passes_job, constant_methods) are explicit parameters rather than extra_options keys
  • Body deliberately raises NotImplementedError: the version this package needs is
    
the multi-graph one (graph-name-keyed dicts, single multi-method .pte for weight
    
sharing), so it lands with the strategy-level fan-out that calls it rather than
    
being written single-graph and then replaced. Inject a custom CompilerAdapter for now.
Config addition:
  • CompilationInputConfig.example_inputs: Optional[Tuple[Any, ...]] — sourced from the model via ModelLoaderAdapter.get_example_inputs
Orchestrator wiring:
  • genai_pipeline.py_run_compilation: passes example_inputs=model_prep_output.example_inputs
Unit tests:
  • test_executorch_compilation_strategy.py
  • test_executorch_inference_strategy.py
  • test_compilation_input_config.py
PR Review Checklist
  • All new classes follow single responsibility (one class per file) - Yes.
  • All dependencies are injected via constructor with sensible defaults - Yes.
  • All external calls are behind injectable interfaces - Yes (adapter pattern).
  • Unit tests cover every public method - Yes (100% coverage on strategy impls).
  • Docstrings on all public classes and methods - Yes.
  • Type annotations on all function signatures - Yes.
  • Logging follows the strategy in the LLD - Yes (info on entry/exit, debug per step).
Related PRs
Test plan
Run only tests added in this PR:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py
-v

Run only this PR's tests with coverage:
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/compilation/ \
backends/qualcomm/genai_pipeline/tests/strategies/inference/ \
backends/qualcomm/genai_pipeline/tests/configs/test_compilation_input_config.py \
--cov=backends/qualcomm/genai_pipeline/strategies/compilation \
--cov=backends/qualcomm/genai_pipeline/strategies/inference \
--cov=backends/qualcomm/genai_pipeline/configs/compilation_input_config \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------
TOTAL 127 0 16 0 100%

Run all genai_pipeline tests:
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Run all tests with coverage:
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing

Result:
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 45 0 10 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 43 0 6 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 668 9 92 1 99%

@pytorch-bot
pytorch-botBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

🔗 Helpful Links
🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22284
Note: Links to docs will display an error until the docs builds have been completed.
⏳ No Failures, 5 Pending
As of commit df39537 with merge base 7f4b8ad (image):

💚 Looks good so far! There are no failures yet. 💚
This comment was automatically generated by Dr. CI and updates every 15 minutes.
@meta-clameta-claBot
added
the 
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Aug 28, 2026
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
@pytorchbot label "release notes: qualcomm"
@pytorch-botpytorch-botBot
added
the 
release notes: qualcomm
Changes to the Qualcomm backend delegate
label
Aug 28, 2026
@psiddh
psiddh
commented
Aug 28, 2026

Copy link


Copy Markdown

Contributor
@claude review this PR
@claude
claudeBot
commented
Aug 28, 2026•

edited

Loading

Copy link


Copy Markdown

Claude finished @psiddh's task in 2m 5s —— View job
PR Review: Compilation & Inference Strategy Implementations
  •  Read changed files
  •  Review compilation strategy
  •  Review inference strategy
  •  Review adapters & config
  •  Post review feedback
Overall this is a clean, well-structured PR. The adapter/strategy separation is consistent with PR5, validation is thorough, error wrapping is correct (StageError re-raise vs. wrap), and test coverage is genuinely comprehensive. Below are the substantive points, in rough priority order.
1. Auto-wired pipeline can never run compilation or inference (design gap)
The orchestrator instantiates strategies with no arguments:
# genai_pipeline.py:163returnstage_cls(strategy_cls())
But both strategies added here depend on an injected adapter that has no working default:
  • ExecuTorchCompilationStrategy() → DefaultCompilerAdapter → compile_model always raises NotImplementedError.
  • ExecuTorchInferenceStrategy() → self._adapter is None → _validate_input always raises StageError.
So a pipeline built via GenAIPipeline.from_proxy(...) — the only public construction path — will always fail at the compilation stage and (if reached) the inference stage. There is currently no route to inject a CompilerAdapter or DeviceRunnerAdapter through from_proxy / _resolve_stage; injection only works if a caller bypasses the registry and constructs GenAIPipeline and each stage by hand.
I understand the NotImplementedError body is deliberate (documented as landing with the multi-graph fan-out), and inference-requires-a-device is legitimate. The gap is the wiring: the DI story terminates at strategy_cls(). It would help to state explicitly whether adapter injection through the orchestrator is coming in PR7, or add a hook (e.g. an optional per-stage adapter map threaded through from_proxy). As it stands the end-to-end path advertised in the PR ("completing the full pipeline") isn't reachable through the orchestrator. Not necessarily a blocker for this PR, but worth confirming the plan.
2. Input data is never pushed to the device
DeviceRunnerAdapter.push_artifacts accepts input_data and extra_files, but the strategy calls it with only artifact_paths:
# executorch_inference_strategy.py:90self._adapter.push_artifacts(artifact_paths=input_config.artifact_paths)
InferenceInputConfig carries prompt and tokenizer, but nothing turns those into device inputs. For a real generation run the model needs inputs on device. Is the intent that the injected adapter fully owns input preparation (prompt→tokens→push) internally, making input_data here dead for this strategy? If so, a one-line comment would prevent a future reader from assuming inputs are wired. If not, this is a functional gap.
3. inference_results type vs. output_data contents
# executorch_inference_strategy.py:109inference_results=result.output_data# Optional[List[Any]]
InferenceOutputConfig.inference_results is typed Optional[List[str]], but output_data is List[Any] and is forwarded verbatim when non-None (the happy-path test even puts ["Hello, world!"] there directly). The fallback branch explicitly str(p)-converts pulled paths, but the primary branch does no conversion. Either the annotation should be List[Any], or the contract that output_data is already decoded text should be documented. Minor, but the two branches having different element types is a latent inconsistency.
4. example_inputs presence check vs. empty tuple
_validate_input checks input_config.example_inputs is None, which is correct given the "required when the stage runs" contract. Worth confirming an empty tuple () is intentionally allowed through (a model genuinely taking no positional inputs) — torch.export would then fail deeper with a less obvious message. Probably fine to leave; flagging only because the config docstring stresses this tuple "defines the exported graph's positional signature."
Minor / nits
  • compile_specs is never populated. In _run_compilation, CompilationInputConfig is built without compile_specs, so it's always None reaching the adapter. Likely out of scope for this PR (they'd be built alongside the real lowering body), but the full path currently has no way to supply them.
  • file_name unused downstream.context.model_name is passed as file_name, but since the adapter body is unimplemented there's no validation that it's sanitized for filesystem use. Fine for now.
  • The docstrings and the deferral rationale (single-graph vs. multi-graph, the rejected recipe-based body) are excellent — genuinely useful context for the next author.
Tests
Coverage is thorough and the tests assert the right things (call ordering, extra_options filtering including the "stale example_inputs must not leak" case, StageError not double-wrapped, two-step output-data-vs-pulled-files protocol). No gaps worth calling out beyond the ones implied above (e.g. no test asserts empty-tuple example_inputs behavior, consistent with #4 being an open question).
Nothing here blocks the mechanical correctness of what's implemented. The main thing I'd want an answer on before merge is #1 — how adapters reach these strategies through the orchestrator — since without it the "full pipeline" is inert.

· branch pr4c
@qti-horodnic
qti-horodnic
commented
Aug 28, 2026

Copy link


Copy Markdown

ContributorAuthor
Addressing Claude's comments:
  1. To clarify, PR7 is for tests only. The wiring will happen in Phase 2 alongside the adapter bodies. Completing the full pipeline might be too strong of a phrase, the strategy layer is complete, the registry path isn't yet reachable end-to-end. The shape of the hook depends on what the adapters end up needing so I'd rather not guess an API before the two callers exist. Today the injection route is constructing the stages directly, which is what the E2E tests in PR7 do.
  2. Intentional. Encoding a prompt is tokenizer- and model-specific, and the runner it wraps already does it on-device, so hoisting it into the strategy would mean the strategy knowing things only the adapter can know. Will add a docstring to clarify.
  3. Valid comment. Keeping List[str] rather than List[Any]: output_data is already-decoded text, the adapter owns decoding for the same reason it owns encoding in (2), so str()-coercing it in the strategy would risk stringifying raw token ids. The fallback branch converts because pulled file paths genuinely are paths. Will add a docstring for this.
  4. Intentional. is None distinguishes "the previous stage produced nothing" from "this model genuinely takes no positional inputs". A () for a no-input model is generally valid for the pipeline. If it's wrong for the model, torch.export is the right place to say so.
@psiddh
psiddh
merged commit c27baa8
into
pytorch:mainAug 29, 2026

190 checks passed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@psiddhpsiddhpsiddh approved these changes
@abhinaykukkadapuabhinaykukkadapuAwaiting requested review from abhinaykukkadapuabhinaykukkadapu is a code owner

Assignees


No one assigned


Labels

CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.release notes: qualcommChanges to the Qualcomm backend delegate

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@qti-horodnic@psiddh