[ET-VK] Optimize conv2d s1p0 by alexdean08 · Pull Request #14187 · pytorch/executorch

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

alexdean08 · 2025-09-11T00:06:43Z

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling.

cc @SS-JIA @manuelcandales @cbilgin

pytorch-bot · 2025-09-11T00:06:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 235536e with merge base 44972ad ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t cc6c080f3a9809a239131be786777a9af99aa9195928b1f4a02f5d5288135a23 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-11T21:24:24Z

@SS-JIA has imported this pull request. If you are a Meta employee, you can view this in D82254889.

SS-JIA

Overall LGTM. Just some questions about things tried and some small requests for stylistic changes. Thanks for working on this!

SS-JIA · 2025-09-11T22:27:03Z

+ outputTexel[3] += dot(inputVec, vec4(weight1OutputChannelPacked[3], weight2OutputChannelPacked[3], weight3OutputChannelPacked[3], weight4OutputChannelPacked[3]));
 }
+
+ imageStore(t_out, ivec3(xIdx, yIdx, gid1), op(outputTexel, out_min, out_max));


iiuc, previously the shader calculated 4 output texels but the new one only calculates one. Have you experimented with computing a bigger output tile? Might be something that can get us a further boost. Note that I am also ok with landing the shader in its current form though, given the perf improvement.

I tried a bigger output tile, and on my end it's slightly slower for MobileNet's case. I believe this is due to the excessive vector registers that are used when we increase the tile size.

SS-JIA · 2025-09-11T22:30:54Z

+ weight4OutputChannelPacked = texelFetch(t_kernel, ivec2(inputC * 4 + 3, gid1), 0);

- const vec4 bias = texelFetch(t_bias, ivec2(out_pos_z, 0), 0);
+ outputTexel[0] += dot(inputVec, vec4(weight1OutputChannelPacked[0], weight2OutputChannelPacked[0], weight3OutputChannelPacked[0], weight4OutputChannelPacked[0]));


In my experience, computing matmul-like operations with dot was not as fast as fma . Did you try computing with fma as well? Curious to know if you have had a different experience. Btw you can take a look at the infographic in the old shader's comments to see how fma can be used instead of dot.

Just tried it on my end. For me it's essentially the exact same for me to use fma instead of dot. Might be some compiler magic happening that makes it be the same operations.

SS-JIA · 2025-10-01T14:07:46Z

@pytorchbot cherry-pick --onto release/1.0 -c fixnewfeature

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin (cherry picked from commit b265324)

pytorchbot · 2025-10-01T14:10:36Z

Cherry picking #14187

The cherry pick PR is at #14724 and it is recommended to link a fixnewfeature cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

@SS-JIA

This change improves the execution of the pointwise conv2d s1p0 shader. It does through more of a GEMM-like implementation and employing more explicit loop unrolling. cc @SS-JIA @manuelcandales @cbilgin

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Optimize conv2d s1p0
8e3d21e

alexdean08 requested a review from SS-JIA as a code owner September 11, 2025 00:06

alexdean08 added module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ release notes: vulkan Changes to the Vulkan backend delegate labels Sep 11, 2025

meta-claBot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

SS-JIA requested changes Sep 11, 2025
View reviewed changes

alexdean08 added 3 commits September 11, 2025 16:13

Stylistic changes to pw conv2d s1p0
41ec8b6

Merge branch 'main' into PWConvOptimized
7f4345d

Add fp16 to conv2d pw s1p0
c1910fe

andreanicastro approved these changes Sep 30, 2025
View reviewed changes

SS-JIA approved these changes Sep 30, 2025
View reviewed changes

Merge branch 'main' into PWConvOptimized
235536e

SS-JIA merged commit b265324 into pytorch:mainOct 1, 2025
129 of 130 checks passed

pytorchbot mentioned this pull request Oct 1, 2025
[v1.0.0] Release Tracker #14288
Closed

xuyanwen2012 deleted the PWConvOptimized branch June 30, 2026 21:58

Uh oh!

Conversation

alexdean08 commented Sep 11, 2025• edited by pytorch-bot BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented Sep 11, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

❌ 1 New Failure

Uh oh!

facebook-github-bot commented Sep 11, 2025

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SS-JIA commented Oct 1, 2025

Uh oh!

pytorchbot commented Oct 1, 2025

Cherry picking #14187

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

alexdean08 commented Sep 11, 2025• edited by pytorch-bot BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented Sep 11, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14187

❌ 1 New Failure

Uh oh!

facebook-github-bot commented Sep 11, 2025

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SS-JIA commented Oct 1, 2025

Uh oh!

pytorchbot commented Oct 1, 2025

Cherry picking #14187

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

alexdean08 commented Sep 11, 2025•
edited by pytorch-bot Bot
Loading

pytorch-botBot commented Sep 11, 2025•
edited
Loading

alexdean08 commented Sep 11, 2025•
edited by pytorch-bot Bot
Loading

pytorch-botBot commented Sep 11, 2025•
edited
Loading

alexdean08 commented Sep 11, 2025•
edited by pytorch-bot Bot
Loading

pytorch-botBot commented Sep 11, 2025•
edited
Loading