[XNNPACK][Weights Cache] Initial Weights Cache Design with NamedDataMap by pytorchbot · Pull Request #9296 · pytorch/executorch

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pytorchbot · 2025-03-14T21:26:52Z

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #9154 by @mcr229
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig
@diff-train-skip-merge

… named data map Pull Request resolved: #9153 We serialize tensors into the named data map, and return the output in preprocess result. Allowing for XNNPACK to share tensors with the same name (instead of duplicating). A key change here is with fused tensors. For BN and Convolution Fusion, we fuse the conv weights and bias with the BN parameters creating new tensors. We then create get_attr nodes for these new parameters. Due to the graph.fx interpreter in export pass base, the new names we create for these new tensors are lost each time. As a result, at the end we introduce a new pass to preserve the names we created. This seems a little hacky for now, but is the only way to preserve the new fused names. Differential Revision: [D70315207](https://our.internmc.facebook.com/intern/diff/D70315207/) **NOTE FOR REVIEWERS**: This PR has internal Meta-specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D70315207/)! ghstack-source-id: 271732046

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

pytorch-bot · 2025-03-14T21:26:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit fa0a9f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

mcr229 added 2 commits March 13, 2025 23:59

pytorchbot requested review from JacobSzwejbka, digantdesai, lucylq, mcr229 and swolchok as code owners March 14, 2025 21:26

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 14, 2025

Base automatically changed from gh/mcr229/9/orig to gh/mcr229/8/origMarch 15, 2025 01:16

SS-JIA approved these changes Mar 15, 2025
View reviewed changes

SS-JIA merged commit d08d938 into gh/mcr229/8/origMar 15, 2025

SS-JIA deleted the gh/mcr229/10/orig branch March 15, 2025 02:31

Uh oh!

Conversation

pytorchbot commented Mar 14, 2025

Uh oh!

pytorch-botBot commented Mar 14, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

❌ 1 New Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pytorchbot commented Mar 14, 2025

Uh oh!

pytorch-botBot commented Mar 14, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

❌ 1 New Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pytorchbot commented Mar 14, 2025

Uh oh!

pytorch-botBot commented Mar 14, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

❌ 1 New Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pytorchbot commented Mar 14, 2025

Uh oh!

pytorch-botBot commented Mar 14, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

❌ 1 New Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

pytorchbot commented Mar 14, 2025

Uh oh!

pytorch-botBot commented Mar 14, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9296

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading

pytorch-botBot commented Mar 14, 2025•
edited
Loading