[XNNPACK][Weights Cache] Initial Weights Cache Design with NamedDataMap by mcr229 · Pull Request #9154 · pytorch/executorch

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 · 2025-03-11T19:54:31Z

Stack from ghstack (oldest at bottom):

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map.

APIs to be used by XNNCompiler:

load_unpacked_data
- Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation
free_unpacked_data
- Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer.
a couple getter methods
- get_packed_data_names
- get_unpacked_data_names
- get_num_packed_data
- get() (get's the xnn_weights_cache object)

Internal APIs used by XNNPACK Library

look_up
- takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists
look_up_or_insert
- takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset
offset_to_addr
- gets offset and returns address to packed pointer
reserve_space
- returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it)
is_finalized
- since this cache doesn't necessarily need to care about a finalized state we always return true.
delete_cache
- deletes cache

Differential Revision: D70885917

XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

pytorch-bot · 2025-03-11T19:54:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9154

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit b5dc2f4 with merge base 630d0cc ():

NEW FAILURE - The following job has failed:

pull / unittest-buck / macos / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-03-11T19:54:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:25:32Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-11T20:40:42Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-12T21:57:37Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-13T22:43:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T07:00:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T16:36:53Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:17:58Z

This pull request was exported from Phabricator. Differential Revision: D70885917

… NamedDataMap" XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/) [ghstack-poisoned]

facebook-github-bot · 2025-03-14T17:31:05Z

This pull request was exported from Phabricator. Differential Revision: D70885917

@mcr229

…ap (#9296) This PR was created by the merge bot to help merge the original PR into the main branch. ghstack PR number: #9154 by @mcr229 ^ Please use this as the source of truth for the PR details, comments, and reviews ghstack PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/10/base ghstack PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/head Merge bot PR base: https://github.com/pytorch/executorch/tree/gh/mcr229/9/orig Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/mcr229/10/orig @diff-train-skip-merge --------- Co-authored-by: Max Ren <maxren@meta.com>

Pull Request resolved: #9154 XNNWeightsCache Design with NamedDataMap. The intent of the weights cache is for tensors to be loaded (via name) through the named data map. APIs to be used by XNNCompiler: - load_unpacked_data - Takes in a string name (tensor name). The weights cache loads the data for this string from the named data map and returns the pointer. It also creates a mapping of this pointer to the name which is later used by the XNNPACK's internal weight cache implementation - free_unpacked_data - Frees all the unpacked data loaded from NamedDataMap. This is only safe to call after xnn_create_runtime has been called. This is because create_runtime takes unpacked data pointers and packs them into a separate buffer. - a couple getter methods - get_packed_data_names - get_unpacked_data_names - get_num_packed_data - get() (get's the xnn_weights_cache object) Internal APIs used by XNNPACK Library - look_up - takes a cache key (weight and bias pointers) and looks up the offset to the packed weight if it exists - look_up_or_insert - takes a cache key and pointer to packed weights and looks_up the offset if it exists, or inserts a new packed weight into the cache and returns that offset - offset_to_addr - gets offset and returns address to packed pointer - reserve_space - returns memory address with appropriate sie for XNNPACK to populate with packed weights ( I want to use the runtime_allocator for this but i don't think we have the right sizes, so for now we are just using a string buffer and resizing it) - is_finalized - since this cache doesn't necessarily need to care about a finalized state we always return true. - delete_cache - deletes cache ghstack-source-id: 271823384 @exported-using-ghexport Differential Revision: [D70885917](https://our.internmc.facebook.com/intern/diff/D70885917/)

mcr229 requested a review from digantdesai as a code owner March 11, 2025 19:54

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 11, 2025

facebook-github-bot added the fb-exported label Mar 11, 2025

mcr229 added the release notes: xnnpack Changes to the XNNPack backend delegate label Mar 11, 2025

mcr229 requested a review from swolchok as a code owner March 12, 2025 21:56

kirklandsign approved these changes Mar 13, 2025
View reviewed changes

mcr229 requested review from JacobSzwejbka and lucylq as code owners March 14, 2025 17:17

facebook-github-bot merged commit dc02cfe into gh/mcr229/10/baseMar 14, 2025

facebook-github-bot deleted the gh/mcr229/10/head branch March 14, 2025 21:26

pytorchbot mentioned this pull request Mar 14, 2025
[XNNPACK][Weights Cache] Initial Weights Cache Design with NamedDataMap #9296
Merged

facebook-github-bot temporarily deployed to cherry-pick-bot March 14, 2025 21:26 — with GitHub Actions Inactive