[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM by JCNTH · Pull Request #20752 · pytorch/executorch

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

JCNTH · 2026-07-06T20:47:49Z

Stack from ghstack (oldest at bottom):

Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express.

Key changes:

q4gsw_linear_gemm_steel.yaml — add a half variant (SUFFIX: half) beside the float variant.
q4gsw_linear_gemm_steel.wgsl — three $if DTYPE == "half" splits (enable f16;, an f16 dequant multiply, an f32(a * b) cast on the MAC so the accumulator stays f32); each $else is the verbatim f32 line, so the float variant regenerates byte-identical.
q4gsw_linear_gemm_steel_half_wgsl.h — generated kQ4gswLinearGemmSteelHalfWGSL (f16 shared memory + multiply, f32 accumulate, f32 storage).
QuantizedLinear.cpp — under WGPU_BACKEND_STEEL_F16, select the half shader only when use_steel and ctx->shader_f16_supported; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only shader_src differs.

The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (add_dtype_suffix).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110802531

[ghstack-poisoned]

pytorch-bot · 2026-07-06T20:47:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

As of commit 061b444 with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 2906ce493c6f45b6362d6845a5b4e911c4aec167905dbb9317c5db59690e5c96 /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 52a6523e572b90533a13849c17245273d0a66a3794c2c99e0e716cef98fe4cc3 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5f6baa7202be54cee596221e2e73f02d2272f193c227c23bb65ce9e3d7b5a016 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / unittest-nxp-neutron / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-06T20:48:38Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JCNTH · 2026-07-06T20:49:49Z

@claude throoughly review, check for overengineering, correctness, and areaas for modularization

claude · 2026-07-06T20:50:41Z

Claude finished @JCNTH's task in 4m 27s —— View job

Review: f16-multiply steel q4gsw prefill GEMM

Read PR diff (QuantizedLinear.cpp, .wgsl, .yaml, generated header)
Verify shader/binding parity and codegen mechanics
Check correctness (f16 mul / f32 accum, enable f16 placement, fail-closed gate)
Assess overengineering & modularization

Overall this is a clean, well-scoped change. The template splits are minimal and the f32 variant provably regenerates byte-identical (the committed q4gsw_linear_gemm_steel_wgsl.h is untouched, and every $else is the verbatim original line). Binding layout, Params, tile constants, and @workgroup_size are identical between variants, so reusing the same workgroup_count/bindings and only swapping shader_src is correct. Not overengineered.

Correctness — looks right

enable f16; is emitted as the first line of the half shader (WGSL requires enable before any declaration) and is absent from the float variant. ✓
f16 multiply / f32 accumulate is implemented as intended: acc stays array<array<f32,4>,4>, and the MAC is acc += f32(a[m] * bvec[n]) — the product is computed in f16, then widened before the f32 add. Never f16-accumulate. ✓
The dequant f16(i32(nib) - 8) * f16(t_scales[...]) is safe on the nib side (-8..7 is exact in f16); scale/activation/product carry the expected f16 rounding, which is the deliberate tradeoff. ✓
The gate is genuinely fail-closed: the half shader is selected only when use_steel && ctx != nullptr && ctx->shader_f16_supported, and shader_f16_supported is set only when WGPUFeatureName_ShaderF16 was required of the device (WebGPUDevice.cpp:151-153) — so enable f16; can never reach a device that rejected it. get_default_webgpu_context() is the same context the graph uses, so the negotiation state is consistent. ✓

Things worth flagging (not blockers)

WGPU_BACKEND_STEEL_F16 has no build wiring.grep finds the macro only at the two #ifdef sites in QuantizedLinear.cpp — nothing in any CMakeLists.txt, preset, or .bzl defines or exposes it. So today the f16 path is unreachable in a normal build; it can only be turned on by hand-defining the macro. That matches the "opt-in, default-off" intent, but consider adding a documented CMake option (e.g. EXECUTORCH_WEBGPU_STEEL_F16) so it's discoverable and testable, rather than an undocumented compile define. Fix this →
No numerical test for the half path. As noted in the description, the golden stays strict-f32 and the half variant isn't exercised. Acceptable while it's off-by-default, but there's currently zero coverage of the f16 kernel's numerics (a tolerance-based golden) — worth a follow-up before anyone flips it on, since f16-product overflow for large activations (f16 max ≈ 65504) is an inherent risk of this path.
Modularization: the 6-line #ifdef override in q4gsw_linear_impl is fine as-is; a select_steel_shader() helper would be over-abstraction for a single call site. No change recommended.

Note on verification

I could not run python backends/webgpu/scripts/gen_wgsl_headers.py --check in this sandbox (command approval), so I confirmed codegen consistency by inspection instead: the half header's structure matches render_header, the float header is unchanged in the diff, and CI's --check will catch any embedded-sha/content drift. If you want, I can add the CMake option from point (1) — say the word.
· branch gh/JCNTH/6/head

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

…ill GEMM Pull Request resolved: #20752 **Enables an f16-multiply path for the 256-thread steel q4gsw prefill GEMM** on devices that negotiated WebGPU shader-f16 — f16 shared memory + f16 multiply with an f32 accumulator (storage stays f32). It is opt-in and default-off, so no ExecuTorch consumer's numerics change; the default build keeps the f32 kernel and the strict f32 golden. f16 multiply / f32 accumulate (never f16-accumulate, which is numerically unstable on the target GPUs) is a measured prefill win the f32-only kernel could not express. **Key changes:** - `q4gsw_linear_gemm_steel.yaml` — add a `half` variant (`SUFFIX: half`) beside the `float` variant. - `q4gsw_linear_gemm_steel.wgsl` — three `$if DTYPE == "half"` splits (`enable f16;`, an f16 dequant multiply, an `f32(a * b)` cast on the MAC so the accumulator stays f32); each `$else` is the verbatim f32 line, so the float variant regenerates byte-identical. - `q4gsw_linear_gemm_steel_half_wgsl.h` — generated `kQ4gswLinearGemmSteelHalfWGSL` (f16 shared memory + multiply, f32 accumulate, f32 storage). - `QuantizedLinear.cpp` — under `WGPU_BACKEND_STEEL_F16`, select the half shader only when `use_steel` and `ctx->shader_f16_supported`; else fall back to the f32 steel kernel (fail-closed). Bindings, tile, and workgroup count are identical — only `shader_src` differs. The device-negotiation gate has no Vulkan analogue: WebGPU shader-f16 is an optional WGSL feature requested at device creation, unlike Vulkan's host-side dtype pick (`add_dtype_suffix`). Co-authored-with: Claude Code. ghstack-source-id: 401515172 @exported-using-ghexport Differential Revision: [D110802531](https://our.internmc.facebook.com/intern/diff/D110802531/)

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Update
161baf5
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 6, 2026 20:47 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 6, 2026 20:47 — with GitHub Actions Error

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 6, 2026

Update
d1ad151
[ghstack-poisoned]

JCNTH requested review from kirklandsign and larryliu0820 as code owners July 6, 2026 21:09

meta-codesyncBot added the meta-exported label Jul 6, 2026

JCNTH temporarily deployed to cadence July 6, 2026 21:11 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 6, 2026 21:37 — with GitHub Actions Inactive

Update
4c20d00
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 03:16 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 03:44 — with GitHub Actions Inactive

Update
642b82a
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 7, 2026 17:59 — with GitHub Actions Inactive

This was referenced Jul 7, 2026
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20770
Closed
[ExecuTorch][WebGPU] Add opt-in native f16 KV cache (−280 MiB, default-OFF) #20772
Merged

JCNTH temporarily deployed to cadence July 9, 2026 04:13 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026
View reviewed changes

Update
523e082
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update
c522131
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

SS-JIA approved these changes Jul 9, 2026
View reviewed changes

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

Update
9a0c964
[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update
061b444
[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:28 — with GitHub Actions Inactive

meta-codesyncBot merged commit 85f8de0 into gh/JCNTH/6/baseJul 9, 2026
177 of 183 checks passed

meta-codesyncBot deleted the gh/JCNTH/6/head branch July 9, 2026 21:39

meta-codesyncBottemporarily deployed to cherry-pick-bot July 9, 2026 21:39 Inactive

pytorchbot mentioned this pull request Jul 9, 2026
[ExecuTorch][WebGPU] Add f16-multiply variant of the steel q4gsw prefill GEMM #20817
Merged

Uh oh!

Conversation

JCNTH commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

Uh oh!

github-actionsBot commented Jul 6, 2026

This PR needs a release notes: label

Uh oh!

JCNTH commented Jul 6, 2026

Uh oh!

claudeBot commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review: f16-multiply steel q4gsw prefill GEMM

Correctness — looks right

Things worth flagging (not blockers)

Note on verification

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

JCNTH commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20752

❌ 4 New Failures, 1 Cancelled Job, 2 Pending

Uh oh!

github-actionsBot commented Jul 6, 2026

This PR needs a release notes: label

Uh oh!

JCNTH commented Jul 6, 2026

Uh oh!

claudeBot commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review: f16-multiply steel q4gsw prefill GEMM

Correctness — looks right

Things worth flagging (not blockers)

Note on verification

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

JCNTH commented Jul 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

JCNTH commented Jul 6, 2026•
edited
Loading

pytorch-botBot commented Jul 6, 2026•
edited
Loading

This PR needs a `release notes:` label

claudeBot commented Jul 6, 2026•
edited
Loading

JCNTH commented Jul 6, 2026•
edited
Loading

pytorch-botBot commented Jul 6, 2026•
edited
Loading

This PR needs a `release notes:` label

claudeBot commented Jul 6, 2026•
edited
Loading

JCNTH commented Jul 6, 2026•
edited
Loading

pytorch-botBot commented Jul 6, 2026•
edited
Loading

This PR needs a `release notes:` label

claudeBot commented Jul 6, 2026•
edited
Loading

JCNTH commented Jul 6, 2026•
edited
Loading

pytorch-botBot commented Jul 6, 2026•
edited
Loading

This PR needs a `release notes:` label

claudeBot commented Jul 6, 2026•
edited
Loading