Arm backend: Allocate the scratch buffer runtime rather than in the pte by gggekov · Pull Request #10714 · pytorch/executorch

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov · 2025-05-06T12:44:53Z

This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

pytorch-bot · 2025-05-06T12:44:57Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job

As of commit 2e023a2 with merge base 2ec8678 ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
pull / android / build-llm-demo / linux-job (gh)
RuntimeError: Internal: unk is not defined.
pull / test-llava-runner-linux / linux-job (gh)
test_llava_export

CANCELLED JOB - The following job was cancelled. Please retry:

trunk / unittest-release / macos / macos-job (gh)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

digantdesai · 2025-05-14T03:53:33Z

 raise RuntimeError("Expected scratch to be int64")
 block_length = int(data["scratch_shape"][0])
- bin_blocks["scratch_data"] = b"\x00" * block_length
+ bin_blocks["scratch_size"] = struct.pack("<I", block_length)


digantdesai

Make sure the CI is green, thanks.

gggekov · 2025-05-14T09:32:34Z

Yes, I am adding support for Dedicated_Sram for U85 & changing the default mem mode we test on U85. This is the proper fix for the fail we see for inception_v4. With this fix, we will place the NN & scratch buffer in the DDR and use the SRAM as a cache. The reason for the failure is that the scratch_buffer for inception_v4 is around 2.6-2.7MB, we allocate the scratch buffer in the SRAM, but on the CS-300 we only 2MB of SRAM. Will update the pr soon.

…Sram for Ethos-U85 This change lowers the size of the pte and allows you to allocate the scratch buffer in an array, usually in the SRAM, for more efficient memory usage on a MCU. Also, add support Dedicated_Sram memory mode in the runtime and make it the default memory mode for Ethos-U85. Change-Id: I04cf9de49a6116141d402b9ad5ca4f21e2025236

zingo · 2025-05-16T15:12:51Z

failed test are unrelated

kirklandsign · 2025-05-17T18:26:51Z

Hi @gggekov @zingo this is breaking our internal test.

undefined reference to `executorch::backends::arm::ethosu_fast_scratch_size'

Seems it's defined in arm_executor_runner only. Any suggestions what we should do on our side? Otherwise OK to revert this?

kirklandsign · 2025-05-17T18:36:46Z

Could you please review #10958 for a temp fix

gggekov · 2025-05-19T10:24:08Z

Hi @kirklandsign,
Thanks for the message, added a comment in #10958

digantdesai · 2025-05-22T16:22:29Z

+ extern size_t ethosu_fast_scratch_size;
+ extern unsigned char* ethosu_fast_scratch;


Ok this is not the cleanest. Let me think of a better way to do this.

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

gggekov requested a review from digantdesai as a code owner May 6, 2025 12:44

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 6, 2025

gggekov added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk topic: not user facing labels May 6, 2025

zingo changed the title ~~Arm backend: Allocate the scratch buffer in an array rather than in t…~~Arm backend: Allocate the scratch buffer runtime rather than in the pteMay 6, 2025

AdrianLundell mentioned this pull request May 12, 2025
Query regarding support of Executorch for ARM Ethos-U65 backend #9356
Closed

digantdesai reviewed May 14, 2025
View reviewed changes

digantdesai approved these changes May 14, 2025
View reviewed changes

gggekov force-pushed the Allocate_scratch_buffer_outside_pte branch from 2a35a48 to 2e023a2Compare May 16, 2025 13:18

gggekov requested review from jathu, kirklandsign and larryliu0820 as code owners May 16, 2025 13:18

zingo added the release notes: arm Changes to the ARM backend delegate label May 16, 2025

zingo merged commit f39a1bb into pytorch:mainMay 16, 2025

digantdesai reviewed May 22, 2025
View reviewed changes

		extern size_t ethosu_fast_scratch_size;
		extern unsigned char* ethosu_fast_scratch;

Uh oh!

Conversation

gggekov commented May 6, 2025• edited by zingo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented May 6, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

❌ 3 New Failures, 1 Cancelled Job

Uh oh!

digantdesaiMay 14, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

gggekov commented May 14, 2025

Uh oh!

zingo commented May 16, 2025

Uh oh!

kirklandsign commented May 17, 2025

Uh oh!

kirklandsign commented May 17, 2025

Uh oh!

gggekov commented May 19, 2025

Uh oh!

digantdesaiMay 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

gggekov commented May 6, 2025• edited by zingo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-botBot commented May 6, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10714

❌ 3 New Failures, 1 Cancelled Job

Uh oh!

digantdesaiMay 14, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

gggekov commented May 14, 2025

Uh oh!

zingo commented May 16, 2025

Uh oh!

kirklandsign commented May 17, 2025

Uh oh!

kirklandsign commented May 17, 2025

Uh oh!

gggekov commented May 19, 2025

Uh oh!

digantdesaiMay 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

gggekov commented May 6, 2025•
edited by zingo
Loading

pytorch-botBot commented May 6, 2025•
edited
Loading

digantdesaiMay 22, 2025•
edited
Loading

gggekov commented May 6, 2025•
edited by zingo
Loading

pytorch-botBot commented May 6, 2025•
edited
Loading

digantdesaiMay 22, 2025•
edited
Loading

gggekov commented May 6, 2025•
edited by zingo
Loading

pytorch-botBot commented May 6, 2025•
edited
Loading

digantdesaiMay 22, 2025•
edited
Loading

gggekov commented May 6, 2025•
edited by zingo
Loading

pytorch-botBot commented May 6, 2025•
edited
Loading